While the rant is valid, I feel like the amount of effort venting about this was equal to or more than actually attempting to get this to work. The author claims that there is "a fair amount of multicast in play which could be part of the issue," but there are no inherent issues with multicast over wifi. My suspicion is that it uses mDNS or some other _link-local_ multicast protocol for discovery. This isn't really news though; any network operator that supports Apple TVs, Chromecasts, etc on their network has had to deal with this (and most vendors have solutions for proxying mDNS).
Proxying mDNS is such a pain if your vendor doesn't have something built-in.
Like when you have to start creating VXLANs to accommodate devices that assume L2 adjacency or installing L7 proxies I think it's fair to say there's some fault with the application vendor.
"IT admin being a dick and won't turn on peer-to-peer networking on the corpnet? Plug a router you bought off the internet into the corpnet tap, or better yet, run a hotspot from some random phone!"
I don't think the title fully encapsulates how astounded I am that this made it through the multiple layers of the Google corporate machine without someone in the chain, someone, saying, "umm..." Next time someone asks you to review those user docs, really read them this time in case you have to be the one raising your hand. ('cuz I know I've been guilty of kind of skimming them.)
According to the description above that section, it doesn't even need an internet connection in that mode, so a quick dumb hotspot for just this purpose seems like a fairly reasonable solution. If IT isn't flexible enough to provide that on request, someone else setting it up at least doesn't hurt that much.
(a server-based fallback would have some value, but I bet people would find something to complain about that too. To slow, unnecessarily sending data back to Google, ...)
so a quick dumb hotspot for just this purpose seems like a fairly reasonable solution.
I believe the issue at hand is the question of who the user is going to call when that doesn't work. The phone vendor? The phone OS vendor? The telecom? Hmm, somehow I feel that those three options will not be first on the list, but rather who is closest and has technical knowledge of any kind.
Who either has "making such things work" in their job description and thus is the right address, or says "not a school-owned device, not my responsibility, please go through the process next time". If it truly works without connection to the internet (and you thus can ignore a lot of the security etc concerns), providing an unconnected AP with a flat network shouldn't be a big issue for most sysadmins. (And yes, I know what first-level support for random non-technical users is like). Yes, they'd prefer if it just worked in whatever environment, but I don't think it's that out of order that it doesn't, especially without knowing the tradeoffs involved in the design.
I agree with the article on "they should specify what they need exactly" though, for those orgs that want to explicitly allow it in their main infrastructure instead of providing a workaround.
I think you're missing an important point, but I don't disagree completely.
The biggest problem is that this is a product DESIGNED to be placed in educational IT environments. As such, the product should either be designed to work in those environments with reasonable effort and without violating common security practices.
It's like marketing a car specifically to people with garages that are too small to fit the dimensions of the car. Either you make the car fit where it needs to go or you market it somewhere that fits. You DON'T market it to a demographic who can't reasonably use it. You're just making bad press for yourself.
And as a network IT guy myself we have a responsibility to maintain order on the networks we administer. Just because someone calls me down to their desk to install CCleaner on their laptop doesn't mean I'm obligated to do it. Infact I'm obligated to steer him in the right direction and make sure he understands the errors of his ways.
IT is the gatekeeper. You don't tell IT what to do... You tell them what you want to accomplish and let them tell you how to do it.
> As such, the product should either be designed to work in those environments with reasonable effort and without violating common security practices.
And I'm claiming that the effort required is reasonable, and it's thus not that big of a deal, even if slightly annoying in that it requires effort and doesn't just work. In the few environments I've worked in a sysadmin role, it wouldn't have been a problem to provide a one-off network that doesn't touch the official network if requested by a user, and in one it'd been completely fine if the user just set up an isolated AP as long as it didn't interfere with the official ones (university network)
The IT department is an enabler for the avarage user or the least common denominator, when we are talking about the monocultures of companies there is no room for niched solutions. The issue here is that ___location based computing is an feature that people need, but the official channels fight.
Companies hire IT departments to manage their IT infrastructure correctly. Maybe some companies just want some cheap kid who can activate a cell phone or plug wires into a tower, but those aren't the companies who NEED to care about IT.
You go to your doctor and pay him good money so he can tell you how to properly take care of your body. If your doctor tells you that you have diabetes and you continue eating sugar, you will die.
If your IT department tells you something is a bad idea there's a reason for it. You don't have to understand his logic, and he doesn't have to explain it to you. Your company pays him to understand all of that so you don't have too.
Enterprise infrastructure is extremely complex and enterprise applications are often extremely fragile. If the company could exist without their contributions I'm sure there would be no IT department.
Bigcorp gets all the positive PR for bringing new technology to schools and "disrupting" the classroom paradigm, while the IT department gets all the blame because Bigcorp doesn't field test.
I don't work in IT, and I'm a bit biased on this topic, but my understanding is that these protocols are chatty and can flood the network on large deployments (e.g. an entire campus that shares the same subnet). They also tie up WiFi airtime. So they're usually explicitly blocked, along with anything else multicast related.
Also, Enterprise IT is often averse to anything that's not centrally managed, and Bonjour is the anthesis of centralization (it's all peer-to-peer). Some networks turn on client isolation so that peers can't talk to each other at all, often in the name of security.
That said, there are ways to get this stuff to work right in an enterprise setting. It requires careful network design to keep broadcast domains small, ideally with an mDNS Reflector running somewhere. All the major WLAN vendors have custom options for supporting it, with varying levels of customizability:
Admittedly, it's hard to get right, the apps that need it are not usually business critical, and there's not many apps to test with... so most IT shops just turn it off and avoid the additional headache. Which is unfortunate, because it effectively kills off any peer-to-peer functionality on the LAN.
Disclaimer: I work at Google, but not on any of the teams mentioned in the article. I am a big fan of mDNS, but that's my personal opinion.
In my experience, it's multicast restrictions. In our case (~8 years ago), the default setup from Cisco was to restrict multicast to within a VLAN. Outside of that, I've seen strict multicast rules that only allowed known services.
It used to be rare for IT to have any real practical UDP/multicast/broadcast experience, mostly because general support was so bad.
I'm no expert, but I think most enterprise & university networks enable client isolation in the routers.
If I know the IP of a friend on the LAN, I can connect to him directly, as normal - the connection will not be blocked by the router. But other than that, the router shows my device a view of the world in which it is the only thing connected to the LAN.
Enumerating the private IP space and trying to connect to all of the possible addresses might reveal the actual stuff though? But the multicast protocols I assume use a smarter and more efficient approach which gets blocked by client isolation.
Client isolation is just wireless clients on the same AP. PIM routing acts like a proxy, and the multicast address for whatever service your routing will be proxied to the clients local multicast address.
Ruckus calls it bonjour fencing, and has pretty good support for creating useful policy.
Client Isolation mode is layer 2 -- its done at the Wireless AP. It should prevent enumeration of the private network space (if the other peers are on the Wireless AP). Neither Multicast nor unicast will work for traffic between clients.
Besides security policy, mDNS uses a link-local multicast address, so routers will not forward it to other network segments. Many operators utilize a number of VLANs on their wireless networks, so this creates an issue when devices in separate VLANs attempt to discover each other over mDNS.
This is usually solved through some sort of mDNS proxy, but you don't really want to proxy everything. If your phone discovered all of the apple tvs and chromecasts across campus, then it wouldn't make for a good user experience for anybody.
My home network consisted of two subnets, and even in such a simple configuration I was never able to get Chromecast working, which also uses mdns.
Mdns proxy would have been the solution, but I was never able to get it to work. It seemed to me that no one actually managed to get it to work, based on my Google searches at the time.
Of course, getting it to work on the corporate network was even less likely.
A subnet or VLAN = broadcast ___domain. Usually broadcast ___domain = multicast ___domain.
Any standard firewall contains tools for PIM (protocol independent multicast) routing. PIM router as the GW for both subnets / VLANs does the trick fine, and is easy-ish to setup. Using this for mDNS printer discovery on a guest network to an internal printer network.
This article is right in that multicast is stupid to use for this, but wrong in that the purpose of a net admin is to NET ADMIN. Make it work and quit whining, multicast is not a new or difficult problem.
Just blocking everything between clients is an easy and strong security policy.
Wireless and wired networks commonly are in different network segments, vs Bonjour etc. generally assuming devices are in the same and using only local multicast - so the network explicitly has to provide a proxy bridging that traffic between segments, not having rules forbidding traffic between them (which also would be normal to have) isn't even enough.
> Just blocking everything between clients is an easy and strong security policy
Or a weak one from the perspective that security must address usability since otherwise users will route around the “secured” system for functions which should be performed within that system.
Multicast on Wi-Fi is quite costly so it tends to be restricted. It is possible to restrict it properly using igmp snooping but there are a lot of rules you have to follow to make it work, it’s easy to make a mistake and typically it isn’t really tested.
Also protocols like Bonjour by default only work on the local network which at home might well be all there is while at work might be one of many.
Deliberate network policies to not forward multicast. So you can only see multicast advertisement on your local segment, and it always seems the device you want to see is on another network segment.
because peer to peer -- ie client to client comms is blocked by the AP for security. Smarter wireless controllers allow selective protocol to pass, so you might be able to allow just mDNS (bonjour). Now this is just for wireless peers, it will allow mDNS to work with other wired clients not using that AP. Its a setting that can be modified.
Making a one-size-fits-badly policy is how you get large amounts of shadow IT and assets on non-controlled machines.
The security policy has to balance with what the users are tasked with, and what's expected. And when IT won't budge, you get really weird stuff happening.
I've seen professors running a linksys natted network on a uni lan, precisely because he needed control and lookup of IPs for his robotics setup. And Uni IT did their knuckle-dragging usual of nothing (blame the user). His solution was "insecure" but that went to his real task of robotics prof.
This sounds to me like the prof tried to tell IT what to do instead of explaining to IT what he wanted to accomplish.
If you came up to me and said "Hey I want to plug in my own router in my office so I can have my own little WiFi network for my projects" I'd tell them no, it's against policy, against SOP, against best everything, and it would be an unmanaged, insecure, non-company asset (probably with default credentials and unpatched firmware). I would then just nod my head at his hemming and hawing and invite him to go over my head.
Now... If you came up to me and said "Hey I've got a ton of wireless devices in my office that are related to my work. What would be the best way for me to network them all together and isolate them from the rest of the network?" I'd gladly draw up a plan to get the task done, order the assets that I want in my shop (with company money), configure it the way I want, and then roll it out and keep tabs on it.
Just doing whatever you want on the network because IT won't let you is asking for trouble, and could/should get you fired.
Yep, and since he was one of the senior faculty teaching engineering (that the uni started the new program for within the last 4 years), he's immune to these kinds of "we'll fire you if you do X it thing".
He needed not wpa2 enterprise wireless. WPA2 personal would have sufficed for his robotics... but "Policies". When he asked how to proceed, IT-Networks responded with "We dont tell people how to do their networking. We enforce policy."
He had contacts in IT (my director), and put me to make it work. So I helped configure a better router, made sure DHCP and other protos wasn't leaking out the WAN, set a stupid long PSK. Took me .5h to get a good environment set up. And sure, it violated policy, but it was secure and enabled the prof to continue his swarming robotics experiments.
I'm curious what's the suggested alternative. If you have a self-setup device with no physical interface, which can be accessible from any place on the network - how would you do that?
Is there some kind of interval version of upnp? Nmb is theoretically possible, not wouldn't work outside of specific environments.
Yes, but this is MUCH easier today. Next Gen firewalls support Deep Packet Inspection for MSRPC traffic, you can even whitelist specific RPC calls, auto/temp allow the ephemeral ports, etc
Yeah, this was a lame rant. Just create a single purpose SSID on your enterprise wireless controller that supports peer-to-peer (client to client connectivity). Bind the educational hardware to that, everything is fine. Really I dont see Google doing anything wrong here...
True that sounds tenable, but dealing with multicast can be a pain. There's no such thing as "enable p2p", there's enable multicast and open specific ports between clients, etc. Does Google give the IP and netmask for the multicast, or the requirements for TTL, or even just the multi-cast ports used, etc? Are they even using multicast? It's all basic info which could readily be listed allowing a configuration to meet the user's needs. Having a single "wide open on all ports" (e.g. consumer grade) SSID on a network could be a significant vulnerability vector.
There is enable p2p, its called disabling client isolation mode. This isn't even specifically about multicast. Its just the peers (clients) cannot make unicast connections to each other.
Buddy, client isolation mode or lack there of is sometimes called allowing 'peer to peer' traffic. I understand the ambiguity over TCP P2P networks, but given the context most network engineers know what you are talking about. Regarding the article mentioning multicast -- the engineer SUSPECTS its to do with multicast. Im saying it could be due to several configuration issues, but most likely client isolation mode (aka not allowing peer-to-peer traffic). Why pick a fight over this?
as far as picking a "fight" I was not, I was being technical. When it comes to talking about Technical things I prefer when people are technical. This is key when writing technical documentation.
For example, if you were writing a technical doc you would not put in the doc "Turn on P2P feature" as that feature does not exist, instead you would say "To enable P2P communication, turn off Client Isolation"
One should be technical when talking about technical things, and assuming "most network engineers know what you are talking about" is often how mistakes are made...
I have seen companies lose millions because one engineer assumed another engineer knew or thought about, or would preform steps not included implicitly in an instruction set
Very likely! But it could scan the subnet, like say a printer driver does. Multicast is the easiest, but not the only way. The issue mentioned in the article is likely related to both mDNS blocking and client isolation (peer to peer).
I have no idea of the peer discovery mechanism in this product -- I was just stating that multicast is not the only peer discovery mechanism, ala printers and other consumer kit that scans the subnet.
I believe one of the selling points of Expedition is that clients can run on students' phones. At that point you have to whitelist hardware of students which come and go, are not managed by you and you experience pain.
Before anyone decides not to read the article because of this TLDR, this is not the summary at all. Which is more, a new Google gadget (Expedition) requires L2 adjacency which is pretty much always blocked in business environments for security purposes. Furthermore, Google suggest ways to get around this...
