Hacker Newsnew | past | comments | ask | show | jobs | submit | jgwest's commentslogin

When setting up multiple VPSs connected by "private networking" with a company like Linode, or Digital Ocean, or what-have-you, you need to assume that the inter-VPS links are not secure. It's a little piece of knowledge that comes with experience, hard for newbies to realize.

The first time you set up one of these clusters, you might follow one of Linode or DigitalOcean's handy guides, where they might suggest i.e. a reverse proxy server receiving (and decrypting e.g. HTTPS) inbound traffic, routing it out to multiple worker machines, and a single backend database system. Linode sells dedicated Load Balancers for the front end of exactly this sort of set-up.

These guides almost always fail to mention that the data is observable as cleartext in the internal network. They ought to be reedited with big bold warnings starting that these links ought to be secured. (Can Linode's Load Balancers even secure these links?) Besides other eavesdropping customers, there could potentially be little magic government agency plugs installed -- or the eavesdropping customers could be government security agencies themselves. (Sorry, tinfoil, I know...)

OpenVPN connections are a lightweight, efficient solution. They're also transparent once you change IPs from those of the virtual network interfaces to those of the secure virtual network interfaces. Such a configuration is still non-trivial, though, for someone configuring their VPS via a control panel rather than the command line.


Hm, I'm not an "expert". Believing that there's any level of security, at this point in time, in any cloud based service is a sign of a semi-retarded IT person IMHO.

That doesn't mean I don't support the cloud, don't use or don't like the technologies that we have today. I wish I could learn and use all modern services out there.

There are various levels of security needed, depending on a case-to-case scenario. I don't believe any company wilfully would create any kind of problems to any of it's users/clients. There are hosting providers (the PirateBay was using one) who are renowned for resisting subpoenas and what not.

BUT if you need absolute security - for whatever reason - in today's world you start by combining carefully the HARDWARE, you don't even buy ready-made products, then you place the server in a place where is physically safe from discrete eyes and hands. Then we can talk software...


People don't "need" to do that, and many don't. They are aided by the false assertions of jackass vendors like this that make false claims further reinforcing their mistaken beliefs.

The configuration overhead of OpenVPN on an internal network like this is lunacy. The correct answer is a secure L2 network... a private one.

Your VM vendor has access to your host ram. Encrypting isn't going to protect you from them. The threat model here is other customers, which are trivial to partition. They just didn't do it.


In your experience, is a single OpenVPN server sufficient on such a network, or is it possible to have a fallback (server)?

I previously tried using a distributed VPN setup without a single main server; that didn't work out so well however, mostly because the software was somewhat unreliable.


The recent past seems like such a dark time for people who are skeptical of all this State Spycraft stuff.

An award to a good film about an essentially important issue, with a little dark (or darkly sinister, take your pick) levity by Mister Oscar, and you start to feel that there is a glimmer of hope...

The EFF can justifiably rant and rave and they do a good job at it. But a little nod from a general public institution seems, to me at least, to provide a special sort of boost.


I think the worst aspect of all these bad actors is how they use misleading language to hide what they are doing.

Consider PrivDog's sales pitch:

PrivDog® protects your privacy while browsing the web and more! Get safer, faster and more private web browsing today!

In fact, the point of the software from PrivDog's perspective is to replace web ads from third-party ad networks with web ads from PrivDog's own third-party ad network -- i.e. AdTrustMedia.

Similar language is used in Lenovo's ex-post-facto sales pitch for Silverfish:

The goal was to improve the shopping experience using their visual discovery techniques.

No, the goal from your point of view was to insert your own advertising network links into user's webpages. And it's installed by default (no need to worry... you can trust your new Lenovo machine!) as a self-encrypted subsystem (which underscores the tricky intentions).

Perhaps the use of misleading language is what primarily leads people to regard these sorts of things as inappropriate bait-and-switch badware installs? The problem is, of course, that these sales techniques work, or at least the offending companies seem to believe that they will work for enough unsophisticated users.


+1 no giant full-width+full-height responsive image

Okay this looks really interesting! And with no full-width+full-height responsive image, I can proceed... it's something different, something not exactly 100% what you would expect...

So intriguing... so what, dear MagicMan, is it? Would you kindly answer dear Sir, because I'm uncertain, and curiously in need of an answer before I text myself down the rabbit hole...

IT JUST LOOKS SO INTERESTING!!! ;-)


I think it's interesting that this BADWARE install was found more or less accidentally... apparently by some tech dude noticing that his bank login presented a Silverfish-issued CA cert.

Shouldn't the possiblity have been forseen and addressed beforehand?

Perhaps by...

(1) Anti-virus / anti-malware makers. Does this software not notify the user when strange CA certs are put into a system's root certificate storage? I understand that certain businesses do this for traffic monitoring... so it might be legit... but still, no user notification?

(2) Microsoft. Do their license terms really allow OEMs to install MiTM proxies and screw around with the root certs? Microsoft could do a good thing here by disallowing this sort of malfeasance... or is there some problem I'm not seeing with such an action?

If this were done in, say, OS X (unrealistic, of course), it would be found out and the whole tech world would know about it in a jiffy. John Siracusa would be howling at the Internet moon within a couple of hours...


I don't know where you got the idea that this got discovered accidentally by this one tech dude. Actually quite a bunch of people have been complaining online about this for months, then for some reason it blew up when the matter got the attention of the tech and sec communities.

see those for example: https://forums.lenovo.com/t5/Lenovo-P-Y-and-Z-series/Persona... http://www.thestudentroom.co.uk/showthread.php?t=3013039 https://forums.lenovo.com/t5/Lenovo-P-Y-and-Z-series/Lenovo-...


(3) Google; Chrome has a rather sophisticated mechanism for detecting MITM attacks, in that it's distributed with pinned certs for several Google properties, and phones home with reports of errors it receives. This is how the DigiNotar leak[1] was discovered.

Perhaps because it was persistent and on the TCP stack level the phonehomes never succeeded? The retry logic should be robust enough to try to deliver the fraud list anyway, even if it will only accept that it has been delivered after a secured connection is restored.

[1] http://en.wikipedia.org/wiki/DigiNotar#Issuance_of_fraudulen...


Chrome does not warn if the non-official root certificate is custom installed on the local machine. It needs to do this because of the various corporate web filters and anti virus tools that MITM connections too.

Maybe this is a practice that needs to stop. Malware scanners can scan on the local machine after the browser has decrypted the communication and web filtering, I think, is nothing but a sign of mistrust against the users.


> Maybe this is a practice that needs to stop. Malware scanners can scan on the local machine after the browser has decrypted the communication and web filtering, I think, is nothing but a sign of mistrust against the users.

It's really kind of a giant security vulnerability. If an attacker can compromise the machine doing the MITM on all the encrypted connections then they get every password and credit card number for every user in your company for every website.


Sure, but once you allow local administrator access to your machine, the "guest" can modify your data and software however it wants, so you've already lost.


web filtering, I think, is nothing but a sign of mistrust against the users.

What if it's the user who wants this filtering?

I run a local proxy that MITMs to filter out ads, tracking scripts, and other undesirable things. It works in all the browsers I use regularly, and any browsers that happen to be embedded in apps, because this way the stuff I want filtered out never even reaches the browser.


Can you please tell more about your setup? Why a handful of browser plugins were not enough in your case?


Filtering reverse proxies e.g. privoxy have an advantage over browser plugins as they work on the network level instead on the DOM. This means that it work as an universal adblock regardless of what OS or browser you are running. It's especially useful when you are on mobile safari or chrome as they don't support adblocks.


The only way Google "needs" to collude with corporate MITM tools is its desire to court user base from corporate IT depts (allowed de jure in many countries that have weak privacy legislation).

Usually Chrome is eager to show security-related notifications but for this there isn't even a yellow notification bar with "OK, got it" option.


I think this is another example of how Google clearly puts its own interests ahead of its users.

Google wants to further promote it's closed Chrome ecosystem, and to do that it needs to gain corporate support, for among other things, its Chromebooks and ChromeOS platform.

And it's obviously more important to appease corporate IT than to protect users security.

Built in Google-spying and now, support for corporate spying too? I wouldn't trust a Chromebook as far as I can throw it.


That's a very impressive case of double think.

Google codes Chrome in order to make it more useful for various kinds of customers, such as customers who have virus scanners.

And this becomes "Google putting its own interests ahead of its users"?

Back here in reality, that's called the customer is always right and is a fundamental tenet of business.


There are many legitimate reasons to MITM web traffic. We don't need to disallow the practice, we need to build a framework that contemplates this need and provides a robust, stable architecture for it which makes it easy to distinguish between good listeners and bad listeners.


Yes, "It needs to do this because of the various corporate web filters and anti virus tools that MITM connections too." <-- this has to go away the sooner the better. Even corporations with a large interest in MITMing their employees (mostly banks, mil, gov) should realize, that this is bad security practice and will lead to all sort of other problems

Banks... imagine the irony.


Perhaps Chrome's MITM detection should only ignore private certs (for web filtering) if configured so via Group Policy or similar mechanism?


> Chrome has a rather sophisticated mechanism for detecting MITM attacks

Which obviously didn't work here, as Chrome was one of the most affected targets.

Firefox on the other hand, was more or less absent altogether. I know which browser I will trust.


Superfish will infect Fx also, it's just that Lenovo didn't pre-install Fx and the installer only runs once.


If you install superfish and then chrome, you will be affected. If you install superfish, then Firefox, you won't.

Thus Firefox is the more secure browser.


Superfish is not a man in the middle, by definition. It's running on your local computer. That's not the middle. That's the start. Consider that Superfish could have just done binary patching on the browser binaries instead of fiddling the local SSL configuration ... it's put there by the computer manufacturer so they can do anything they like.


It's called a "man in the middle" because it intercepts connections between the source and destination. The physical location is irrelevant.


That list is public; if you are in the business of writing these proxies anyway, fetching that list and using it as do-not-mitm exceptions is not a stretch. Which, unfortunately, defeats this nice side-effect of certificate pinning. People could have learned from the Diginotar mistake (being: mitm'ing ssl-pinned certs).


> (1) Anti-virus / anti-malware makers. Does this software not notify the user when strange CA certs are put into a system's root certificate storage? I understand that certain businesses do this for traffic monitoring... so it might be legit... but still, no user notification?

It was installed by the OEM. Doesn't really help if it only notifies the OEM.

> (2) Microsoft. Do their license terms really allow OEMs to install MiTM proxies and screw around with the root certs? Microsoft could do a good thing here by disallowing this sort of malfeasance... or is there some problem I'm not seeing with such an action?

The general solution to what you're talking about is to prohibit the OEMs from installing anything by default. The problem is the OEMs wouldn't like it and Microsoft has to keep the OEMs happy lest they get any bright ideas about offering their computers with Ubuntu for $50 less than Windows.


It's not just that the OEMs wouldn't like it. The US DoJ sued Microsoft (and tried to break it up) to prevent it from having any control over what they do. In fact, Microsoft doesn't know what OEMs are installing as "Windows" unless it goes out and buys one of their PCs.

Otherwise, some OEMs have tried installing versions of Linux, with negative financial results. A few are still trying. The real problems are selling and supporting them.


> Otherwise, some OEMs have tried installing versions of Linux, with negative financial results. A few are still trying. The real problems are selling and supporting them.

The problem seems to be that they're always trying to put them on budget machines, which is completely the wrong market. It's chasing the customers who pinch the last penny and you're never going to make any money from them regardless. Meanwhile those customers don't know what an "Ubuntu" is but pick it because it's cheaper, and then you get overrun with support calls when they want to install Turbo Tax.

The place where it makes much more sense is the corporate and professional markets where the customers actually know what they're buying. An IT department which is just going to nuke whatever the OEM installs in favor of their own volume licensed disk image would be happy to save the cost of a [redundant] Windows license for every machine. And professionals like programmers and scientists who actually use Linux would appreciate being able to buy workstation-class hardware with official driver support.


You are exactly right on both counts.

The main attempt to sell Linux to end users was the use of different versions on netbooks, which were mainly bought on price by relatively clueless users.

I talked to one supplier about the obvious cost-of-Linux-support problem at their launch. We won't do support, they said, it will be like an appliance: we'll just reset to factory condition.

You can imagine how that turned out...


>some OEMs have tried installing versions of Linux, with negative financial results.

Which isn't much of a surprise considering what I have observed so far (in trying to purchase a Linux PC). I can't recall ever having seen an OEM offer Linux for more than a sparse subset of their product line, usually mid-tier or low-tier machines.

>A few are still trying.

Which ones? The situation may have changed since I last paid any attention a few years ago.

>The real problems are selling and supporting them.

The MVP here is to merely accept returns for units that turned out to be particularly troublesome; which they usually do (ie: the Samsung UEFI thing from 2013).

A non-Microsoft UEFI key thing might be nice as well, but that's another story.


Wal-Mart sold Linux machines at one time, and maybe still does. Dell does. A lot of small suppliers do (because they don't get such big OEM discounts on Windows and don't have high-volume automated production lines). But the real problem is that one "support incident" eats the profit from about five sales, or more.

If you think there's a market for Linux PCs, you can always set up a company to sell them. You wouldn't be the first to try, but you might be the first to succeed ;-)


>Dell does.

Dell used to. I just contacted Dell sales and according to "Hazel" they do not offer any non-Windows OS for consumer products nor will they sell a system sans-OS.

>But the real problem is that one "support incident" eats the profit from about five sales, or more.

Meh, there is a lot of room for argument here. I think the real problem, after MS' many anti-competitive shenanigans is that most people just think MS Windows is synonymous with "computer". Those who really want a Linux PC will just buy the hardware they want and install it themselves.

>If you think there's a market for Linux PCs, you can always set up a company to sell them. You wouldn't be the first to try, but you might be the first to succeed ;-)

Someone someday will probably succeed at that. I'm probably not that someone, and that day may not be today. I do think that there is a small market for it, and there could be a bigger one, maybe if/after Gaben has any success with SteamOS. OTOH, if we ever have a modular laptop standard with a commodity peripheral market then maybe not, as there would be less need. (given that the only OEM pc's I have purchased in the last 10 years were laptops).


> Meh, there is a lot of room for argument here.

Not really. I got my info from senior managers at some of the (very large, Taiwanese) companies concerned.

> I think the real problem, after MS' many anti-competitive shenanigans is that most people just think MS Windows is synonymous with "computer".

Microsoft has never been accused or taken to court for any "anti-competitive shenanigans" re the success of Windows, only for ways it tried to exploit that success.

> Those who really want a Linux PC will just buy the hardware they want and install it themselves.

Yes, exactly. And they will install whichever of the 157 versions they prefer. These are among the reasons why it's hard to make a profit selling Linux PCs.


>Not really. I got my info from senior managers at some of the (very large, Taiwanese) companies concerned.

Oh. Why didn't you tell me that you had a real authoritative source? /s

>Microsoft has never been accused or taken to court

Microsoft's historical business practices WRT both Apple and Linux are well documented. Lawyers have made whole careers work generated. There is no need to hash this out for the billion +1th time.

>These are among the reasons why it's h...

I don't know who you think you're arguing with. I haven't asked you for seed money. I've not tried to convince you to go into business selling PC's.


> Oh. Why didn't you tell me that you had a real authoritative source? /s

Sorry, reality intruded. I should have known you'd find that a problem. However, you could get a clue from the fact that most companies who have tried to sell Linux have either stopped or gone bust, or do it on a very small number of systems. This is not because they are against making a profit.

> Microsoft's historical business practices WRT both Apple and Linux are well documented.

Up to a point. But most of the inexpert comment I see is badly informed and usually wrong. Still, who reads documentation?

> I don't know who you think you're arguing with

My mistake, I didn't know I was arguing....


> Dell used to. I just contacted Dell sales and according to "Hazel" they do not offer any non-Windows OS for consumer products nor will they sell a system sans-OS.

They do, it's called "Project Sputnik". It's targeted at developers though, which is a market that clearly makes sense, as AnthonyMouse pointed out.

http://www.dell.com/learn/us/en/555/campaigns/xps-linux-lapt...

The XPS 13 review yesterday was interesting, but I think I need a more beefy machine. Anyone has experience with this precision developer edition on Linux?

For a company specialized in Linux PCs, there is System76.


Thanks for the links. I confined my statement to consumer products, so that's still true. I am a little disappointed that "Hazel" didn't mention this one, because I specifically asked about XPS series laptops, and then asked if there were any other.

These days I have a purchasing department that impedes my purchases, so as long as the crap they give me isn't too bad I just let it be.


... or they could develop badware for Ubuntu.


I found it by myself several weeks before all this news came out.

I got my new Lenovo Y50, visited my own website with it and decided to see how my https cert looked. I got quite scared when I saw I was being MITMed but I googled it and there were already a ton of forum posts saying it's just stuff bundled with Lenovo. So I uninstalled it.


Note that uninstalling the program doesn't completely undo the damage; you also need to get rid of the trusted certificate that it uses to make all of its forged certs look legitimate to the browsers. (The private key for that cert has been widely distributed, and at this point, anyone can use it to make a cert for your bank that will look legitimate to your machine so long as the Superfish root cert remains in place.)

Complete instructions here: http://www.pcworld.com/article/2886278/how-to-remove-the-dan...


Maybe it's the doing of the U.S. gov't... maybe not...

But in any case, what's the point of keeping the U.S. government's action or non-action secret?

As the linked piece states:

"If the attack was American in origin — something the United States would probably never acknowledge ..."

It's sort of like the Doomsday Machine in Dr. Strangelove: it just doesn't work as a deterrent if you keep it a secret.

Or is all this secret "cyberwarfare" capability that the U.S. government is secretly building only going to be used in secret?


Your excellent point has a follow on: who exactly is responsible for deciding when or if such a retaliation is to occur? Is there any oversight? Is there any accountability? Who are the individuals involved? Which agency is involved? Is Obama fessing up or not? I am not saying that I agree or disagree with the retaliation, only that open accountability is necessary, precisely such that our collective liberty is safeguarded.

If this is not the US, then the cybersecurity apparatus of the US and other nations must surely provide more information about which entity has the power to take down an entire country's internet (even if, admittely, this is a small country that is easy to take down?). We need to know either that this is an explicit retaliatory attack (in which case, who is deciding the legitimacy and proportion of this retaliation), or if not, we need to know very clearly that our cybersecurity apparatus is aware of who did it, and if not, what are they doing to become aware of such issues in the future (with guarantees of public disclosure when this is not incompatible with national security).

Basically, we cannot have a situation where signficant swathes of the internet can be taken down with nobody knowing what's going on, and what the principles are behind any decisions made. That would be a basic affront to freedom.

Nebulous, intangible entities with the power to perpetrate or retaliate with no accountability, are extremely dangerous.

I see a significant dearth of information here, information that is in the public interest whoever is behind it.


Supposedly CloudFlare has ways to trace and then block a DDoS attack. So, maybe they know the origin of the NK DDoS attack if it was a DDoS.


i would suspect that if it was a retaliation and secret to the public, it would certainly not be secret in negotiations and what not.


Dudes... I don't really care... I just want a way to synchronize my VPS clock with that of other, established, secure clocks... because DigitalOcean (okay, go ahead and downvote me) is not quite synched, and neither is Linode, and my server!!! Oh good Lord my server don't know the time at all! Dudes... just agree on something that I can install SIMPLY... 'cuz the infighting between ntp and openntp ain't nothing that I care to be involved with... JUST MAKE IT EASY!!!!


What are you whining about? What DO, Linode, etc. Have to do with anything????

OpenNTPd is extremely easy to configure. The problem is that it's not the default choice while, apparently, it should be. Same goes with almost all OBSD-derivates (OpenSMTPd, OpenSSHd, OpenNTPd, etc.).

Here is the first guide at google: https://wiki.archlinux.org/index.php/OpenNTPD

Feel free to ask, if you feel that there's something exotic that needs to be done before getting OpenNTPd to work.


> What are you whining about? What DO, Linode, etc. Have to do with anything????

Well, there's a good argument to be made that your VPS provider should be running an accurate time daemon (whatever that might be), and you should just have paravirtualized access to the current time, instead of every VM being expected to run some form of ntpd on its own, and the hypervisor track scores of virtual clocks.

There's even a good argument to be made that running the NTP algorithm in a VM doesn't work: https://www.usenix.org/legacy/event/osdi10/tech/full_papers/...


Oh I see. I didn't knew that, I just connected the NTPd daemons either to the default NTP server or to the ISPs/Provider's NTPd server. Hm, didn't knew that VM had such issues with time servers.


It's also documented best practice, at least with VMware and Xen.


Okay, Brittney Bronson, it is what it is... You teach the young grown-ups / old kids at the college, and then you shuck it in your part-time service job... It is what it is...


Too much condescending tone in this "proposal"...

I think people just want to see the film. I certainly want to see the film. It might have a crappy plot or a second-rate screenplay or subpar acting, but with this sort of publicity none of that matters. Just watching it will be an event, perhaps an even bigger event than watching The Last Temptation of Christ was way-back-when...

Anyway, Sony seems to be in a defiant stance. It doesn't seem like Sony is going to yield; it seems like they are going to just find an alternative distribution path: "No thanks, 2600. We got this. After all, this is the sort of hype that we'd... uhh... kill for.... uh..."


To be honest, I have no interest in seeing the film because everything I've heard about it indicates it's probably a crap film. I furthermore think it was a bad idea to make it in the first place.

However, I applaud 2600's proposal because it points in the right direction: a calm, nonviolent but assertive refusal to be intimidated, rather than the hysterical paranoia and escalation of threats we are seeing all too much of in recent years.


>Just watching it will be an event

That brings up a good point. Perhaps Sony should even consider using the controversy in its marketing, then later make claims about how much more successful the film was than projected, thanks to the added publicity (which will almost certainly be true).

Not only would it show defiance, it would underscore the paradoxical effect of trying to stifle free speech in this manner. And, that might provide the biggest disincentive of all for future prevention: demonstration of ineffectivness.


What has lead you to believe that free speech is any consideration in Sony's dealings? This is the company that has (and still does) push for SOPA and CISPA-like abilities to take down websites with the most minor effort!

It's unclear why the release was canceled. Maybe Sony wants to just put this whole thing behind them. Maybe they want to curtail any further leaks that they feel may be worse. Perhaps the executives feel that being a victim of a foreign nation absolves them of any culpability and they're playing that card to the greatest extent they can. But I highly doubt they're going to make any about face on a policy issue and are still very much against free speech as far as the internet is concerned. If anything this incident will be used to bolster their arguments and to that end maybe it has worked out better for them than "The Interview" ever could.


>What has lead you to believe that free speech is any consideration in Sony's dealings?

Call it artistic freedom if you'd prefer. It's all a form of free speech and I'm not sure that Sony has ever come out against artistic expression.

Regardless, I think it's pretty clear that the shoe is on the other foot now. That point, of course, is essentially the main premise of the article on which we are commenting, and it's what makes the situation so sweetly ironic to its author(s).

But, you know, it's all P.R. and all about messaging (from both sides). Sony doesn't have to really believe in free speech for everyone in order to make a stand on that premise. They are surely aware that standing on their right to make a profit probably wouldn't engender as much support or be as effective as standing for artistic freedom, free speech, etc.

And, of course, all of this discussion about what to call it is pedantic, because my bigger point was that they have an opportunity to turn this on its head and have it completely backfire on the attackers, thus providing strong disincentive for future attacks.

>It's unclear why the release was canceled.

I think it's pretty clear that, at least in part, they were tired of having their asses handed to them. In short, they were punked.


Seems like somebody in the DoJ just decided that Tor's balance between geeky CompSci curiosity and enabler of real-world criminal behavior has tipped too far in the latter direction. The legal case has been ripe for a while-- after all, Megaupload and many other networks have been disabled by the US government for enabling significantly LESS serious criminality. Ummm... world's biggest drug marketplace, anyone??? What's important to remember is that the gov't can't just go in and seize the directory authority servers willy-nilly. Instead, they must do it as part of a legal process against a specific, identified target. In this case, the likely target is going to be the Tor project itself and possibly the individuals leading it. The legal case might ruffle a few techie feathers but only an insignificant portion of the general public will care, and that portion can be mollified with the "stopping the bad horrible criminals" routine.


Those were not shutdown for "enabling" criminal activity. They were shut down for actually doing criminal activity. With megaupload it was failure to abide by the DMCA, with silk road it was handling money for/from drug dealers. I cannot see how Tor has actually done anything criminal beyond what a thousand other transitory service providers do every day.


in case of megaupload it was totally ILLEGAL, using bogus charges


According to the internal emails the prosecutors got their hands on, megaupload was paying out to the top uploaders, and megaupload showed knowledge of what the uploads where. I can't stand commercial piracy.

http://www.justice.gov/usao/vae/victimwitness/mega_files/Meg...


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: