This is a good thing. When they were independent, NPM was a disaster area. The company spent 100% of its time chasing down social issues and insanity in the community and never figured out how to make money, or at least, it took them FOREVER to figure that out.
Years ago, they introduced "orgs" which they sat there and explained to me with slides and pictures and concepts and business bullshit for an hour. I did not understand a thing they'd said. Finally, they were like "We're selling private namespace in the npm registry for blessed packages for groups or businesses." I understood that. If they'd just said that up front....
They had some great people, some very smart folks like CJ, but they completely biffed every business decision they ever made, and when you'd go in and talk to the leadership, they were always acting as if they had some sort of PTSD from the community. I mean, people were putting spam packages in NPM just to get SEO on some outside webpage through the default NPM package webpages. People were squatting and stealing package names. Leftpad... the community management here is nightmarishly hard, and I was never convinced they'd ever make money on it. MS doesn't NEED to make money on it. They can just pump in cash and have a brilliant tool for reaching UX developers around the world, regardless of whether they use Windows or not.
I feel like the GitHub group at Microsoft is now some sort of orphanage for mistreated developer tool startups. GitHub had similar management issues: they refused to build enterprise features at all for years unless they were useful to regular GitHub.com. And there were other people issues at the top for years. Chris seemed more interested in working with the Obama administration on digital learning initiatives than with running GitHub, for example.
> I mean, people were putting spam packages in NPM just to get SEO on some outside webpage through the default NPM package webpages. People were squatting and stealing package names. Leftpad... the community management here is nightmarishly hard
It's not NPM's fault (well, other than wrt. the leftpad thing), it's all about the "community". The Javascript open source community is a dumpster fire.
Oh please... I’ve solved more customer needs and business value from this "dumpster fire" than I have with other "real languages" I've worked with in my career. Give it some credit.
It's like modern fighter jets which are designed to be unstable so they are more manoeuvrable, aided by the computer controls to stop that instability being dangerous. If the computer systems fail, the pilot will not cope and the jet will crash and burn.
The JS community is unstable, but in a way that produces useful things rapidly. Stuff just gets thrown out there and the good parts stick. Certain dampening factors protect you from the most dangerous effects of this (unless you live your life at the bleeding edge) like the jet's computer controlled micro-interventions protecting the pilot form how jittery the airframe actually is. Occasionally this goes wrong and the protection fails. "Leftpad" was one such matter that caused thing to crash and burn temporarily. Luckily unlike an expensive jet hitting the ground, the damage from such incidents is relatively easy to repair after the fact.
The throw-it-out-and-see-if-it-works part isn't the only issue with the JS community but it is a significant one and does relate to some of the others, though one that does provide some benefit in the form of forward momentum.
Their productivity is directly related the to vast ecosystem of packages available. Those packages were written and maintained by the community. They are rebutting the assertion that this community is bad. It can't be that bad if they're helping millions of developers solve real problems around the world.
I don't think it's fair to victims of abusive relationships for you to use this analogy: the truth is far darker and more complex, and survivors deserve more credit than you're giving them. I could give you my story as an example in case you haven't ever been exposed to this kind of abuse, but maybe it is enough for me to say we should all think twice before using such an extreme and unique scenario in our arguments.
As an online community gets larger, the probability of a member deserving a comparison involving Nazis or Hitler approaches 1?
That the JS community is huge and overwhelmingly newer to the industry than (for example) me does not excuse bad behaviour. What it _does_ excuse is a lack of awareness of history and historical context. And the solution to that is education.
It pains me to see so many wheel-reinventions, both technical and social. But the flip side is that people who don't know something's not supposed to be possible will _sometimes_ manage to do it anyway.
NPM, on the other hand, seemed very much to be all about package management being a simple problem, and learned the hard way that there's a reason why other systems have more complexity.
They could have made a LOT of money (I think) by offering a "secure" registry - only packages that were reviewed, verified and signed would end up in there, and they would be sealed and made available forever. Companies could have their developers use only that because there is still a huge risk of a malicious actor pushing a patch version of a library with a security vulnerability in it, and at the moment security is still a reactive action in the Node ecosystem.
What companies do you see paying for this service? Why would they prefer paying a company for this over a community-driven effort to curate secure packages.
> I feel like the GitHub group at Microsoft is now some sort of orphanage for mistreated developer tool startups.
That seems .. not so bad really? Microsoft gets to buy them cheaply, and they don't get obliterated or acquihired, and they don't seem to have Yahoo'd them into slow death either.
In an existing UI project repo... ci (which clears node_modules) then installs from lock...
added 1880 packages in 23.283s
real 0m24.073s
user 0m0.000s
sys 0m0.135s
Still slower than I'd like... but I'm pretty judicious in terms of what I let come in regarding dependencies. That's react, redux, react-redux, material-ui, parcel (for webpack, babel, etc) and a few other dependencies.
For one of the API packages ci over an existing install...
added 1069 packages in 12.911s
real 0m13.708s
user 0m0.045s
sys 0m0.076s
So either you're including the kitchen sink, or you're running on a really slow drive.
Was using a hackintosh and rmbp until about 2 years ago, stopped using mac at work, and in october switched to a new desktop and jumped to linux. Been back in windows + wsl2 for a couple months now.
Back using mac, and most of my windows until a couple months ago, was still mostly linux via VM.
Guess I never realized how slow macos's file system was for deleting files.
edit: Also, for those curious, WSL2 files in Windows is slow, and windows files in wsl are slow... each are fast in their own sandbox.
It's not a bad idea for MS to sponsor this stuff as a PR play, but it needs a strong commitment to ombudsing and community oversight, or it becomes just another Embrace/Extend/Extinguish.
Someone asked "Would this have made sense for a company like GitLab if they didn't have the corporate backing of something like MS?" and deleted their comment while I was writing the answer below:
Being the canonical registry for a language (Rubygems) or technology (DockerHub) tends to be a huge expense.
The main expenses are cloud costs (bandwidth and storage) and security (defense and curation).
I've not seen examples of organizations turning this into a great business by itself. For example Rubygems is sponsored by RubyCentral http://rubycentral.org/ who organize the annual RubyConf and RailsConf software conferences.
Please note that running a non-canonical registry is a good business. JFrog does well with Artifactory https://jfrog.com/artifactory/ and we have the GitLab Package Registry https://docs.gitlab.com/ee/user/packages/ that includes a dependency proxy and we're working on a dependency firewall.
It's a mystery how DockerHub remains free. The network and storage costs must be massive. Docker also sold it's enterprise business, so I am not sure who is paying for all this.
If you're running a package registry, you'd be painfully unaware of your own requirements if you're using a host/cloud that you need to pay for the amount of bandwidth.
Hosting a package registry on AWS for example, unless you figure out a way of seriously reducing the amount of traffic (which seems to be against working towards making a popular registry), is suicidal because of the bandwidth costs.
This massively depends on the architecture of the package manager though. Composer (PHPs primary package manager) only serves the metadata and directs the installer to the already packaged ZIPs hosted on Github/Bitbucket/Gitlab. This allowed them to serve the whole community as a small team of 2-4 for almost a decade now without any outside funding.
> This massively depends on the architecture of the package manager though. Composer (PHPs primary package manager) only serves the metadata
See also, CocoaPods. But we're really talking about two things here as if they're the same, and I think it's because the language we use for them isn't clear: package managers (like CocoaPods and Composer) and full-blown package registries (like RubyGems and NPM). I mean, the difference between them is slight; and there's overlap. They both resolve package dependencies and sort out problems, for example. But they're such very, very different things!
It's not really fair to compare these two approaches and say things about Composer like "this allows them to serve the whole community with a team of 2-4 people": they're just not directly comparable. Those 2-4 people + the entire GitHub staff is what supports the whole community.
Now - don't get me wrong: I think it's fine that GitHub does this! I'm not saying that CocoaPods and Composer are making the wrong choice here at all. In fact, I'm proud to work for a company that provides this service to everyone. But it hand-waves away a lot of stuff as just an "architecture" decision, when in reality it's a decision not to do the hardest part - which is hosting, because it requires gobs of money and dedicated staff and people on-call 24/7.
Framing it as just a different "architecture" implies that registries like NPM are making the wrong choices by hosting their own downloads, which I think is not nearly that simple. Because what happens if GitHub or Bitbucket decided to not allow this type of usage? It would be devastating for the community - so I think it's very appropriate for package managers to make a decision about whether or not they also want to be a registry. They're considering what could happen down the road and what that would mean.
Speaking of... The NuGet registry in GitLab deserves an option to run without authentication from other builds and/or the "public" accessing it. Currently, it does not even handle the build job token and I have to cross connect builds using my personal access token (which then again is accessible to other maintainers of a repo).
Yep, non-canonical registries are used by business and they are more likely to pay. For example our dependency firewall will be a paid feature that delays updates from packages that are recently updated under suspicious circumstances like: author changed, author information changed, different programming language, activity after a long period of inactivity, large change to the code, etc.
> delays updates from packages that are recently updated under suspicious circumstances
This is an example of the security curation you note earlier.
People reviewing flags alongside the code, remediation during the delay, highlighting package authors you like are some of why “Being the canonical registry … tends to be a huge expense.” How many people build and run GitLab Package Registry?
We don’t know exactly how many people use it since it is open source but I think between 100,000 and 1 million people. The docker container registry is the most popular.
The security curation examples I mentioned are intended to use automated signals so there wouldn’t be an expense for labor. In fact we intend to run them separately on every self managed GitLab installation.
Can anybody comment on how hosting a package repo for someone large may help peering? I'd imagine they would see significant upstream traffic and globally but not certain if that makes an impact
I think hosting a package repo will generally create more downloads (costs money) than uploads (helps with peering but costs storage) because the average number of downloads for a package is more than 1.
I never quite got a warm-fuzzy feeling from npm -- the tool, the service, the company. This announcement does nothing to help, from my perspective. Is my dependency on this or that JavaScript library something that really needs to be owned by a for-profit company?
I also kind of wonder what is the real value of a centralized repository versus just directly referencing git repos. I haven't used this gpk[0] project yet, but it looks like an interesting alternative, on paper.
Much better: mandatory vendoring of packages. Can't break and being forced to push the packages to the repo makes you appreciate the lack of transient dependencies.
This brought up in my mind the thought that while Deno is still WIP (for example, packaging of Rust plugins is not yet resolved) and the ecosystem around it barely exists it was designed to have no dependency on 3rd party tools like npm and yarn.
I wasn't aware of the semver URL scheme, but it just brings more problems.
At this point they've just replaced `npm` with `pika` and expect people to think it's an improvement to not have a centralized file containing package dependencies and the last resolved versions of packages. Criticisms:
1. Updating dependencies becomes much more of a diff (every file that uses it needs to be updated). The alternative solution to this is creating a single `packages.ts` file that imports all the external deps, and all your modules import from there. Which just `npm` with more steps.
2. No lock file. I saw them present this at tsConf and I believe the original idea was that this wasn't needed because the URLs are the lock, but as you point out in practice people use URL's that aren't unique identifiers. This means no way to guarantee that all developers or even users of the project have the same dependency state. This will be a massive pain for debugging and maintaining packages. (Edit: see below, you can lock deps (unclear exactly how) - but only if you pay them!)
Looking at the Pika home page, they tell me I can build and release without a bundler.... so you're telling me that they expect people to put out production software that downloads it's dependencies at runtime and thus:
- Is unstable because if they use the semver-URL scheme you mentioned and a patch version comes out that breaks the website, all users are instantly broken instead of the internal build being broken.
- Is unstable because if a dependency/host goes offline, all users are broken instead of an internal build being broken (think leftpad but much worse as your users are instantly impacted, likely before you even know about it)
- Is insecure because if a host is malicious, they can choose to supply different packages for a small subset of the requests, such as those coming from govt. requests against political targets, hosted build machines, etc. and nobody will have any way of knowing because there's no lock file/integrity hashes.
I further see on the Pika CDN page that they share packages across websites. This is seems to be a massive security flaw, as websites are able to modify these packages and now those modifications will apply to all websites using the package. It's prototype pollution-as a-feature!
Oh, and at the bottom of the page:
> Want to get more out of Pika CDN? The CDN will always be free, but you can also access paid, production features like:
> Granular Semver Matching & Version Pinning
> ...
So base features that are the core of working with npm/yarn/any modern package manager are a paid feature in Pika.
That's mostly unintended FUD (but still FUD) or problems that can be worked around:
<<
- Is unstable because if a dependency/host goes offline, all users are broken instead of an internal build being broken (think leftpad but much worse as your users are instantly impacted, likely before you even know about it)
>>
So you don't recall Leftpad?
<<
- Is insecure because if a host is malicious, they can choose to supply different packages for a small subset of the requests, such as those coming from govt. requests against political targets, hosted build machines, etc. and nobody will have any way of knowing because there's no lock file/integrity hashes.
>>
Pika is just some website serving js.
deno is including a bundle command that will create a single script that contains all your deps that can be version controlled and/or distributed.
also, even without bundling deno doesn't download imports every time it is run. It downloads imports when you first run it or when you ask it to --reload. Seems quite similar to npm in that regard, except for the lockfile perhaps.
They are also working to support importmaps which could become part of this story.
In any case, competent and smart people are working to make it usable. It is still a work in progress but I love the direction they are taking. It will be great to be able to run safe javascript where I provide it with the permissions it gets.
NPM is both a massive repository as well as a package manager.
Deno will (soon?) have a package manager, but it won't be tied to one central repository run by a private company, now part of a massive corporation (Microsoft.)
So let's not lump package manager and repository: you can have a package manager that pulls in all the exact or latest dependencies at build time, but it does not have to be tied to one central repository owned and administered by one company.
It's not about choosing a different registry for all your modules, which will almost always be a commercial for-profit entity due to the cost of running and maintaining a centralized repository of that scale. AFAIK, or as far as I can envision, it's about a future where we can have the repository be a decentralized graph. It's definitely more challenging but we won't be dependent on one or few companies to host all modules. What do you think of that idea? My point is Ryan was right to avoid recreating dependence on commercial for-profit entities.
That's quite strong wording, nothing in that article indicates that Deno is blessing Pika specifically.
In fact Ryan Dahl said one of their biggest regrets was blessing npm in node. I'd be surprised if Deno jumped into blessing a package registry any time soon.
Out of curiosity, what benefits does Microsoft/GitHub get from owning a package registry? I'd be fascinated to learn more about their long-term strategy here.
Others have commented on why the acquisition makes strategic sense from a technological perspective, but I think it's also important to consider how it makes strategic sense from a psychological perspective. For a long time, devs loved bagging on Microsoft. I once saw some Microsoft guys demo something cool at a conference, and they had to basically apologize that they were from Microsoft, because dev sentiment toward the organization was so negative, even though they were doing cool stuff.
The acquisition of GitHub was absolutely intended to capture a tool/ecosystem that developers liked using and benefit from that positive sentiment. That's why Microsoft has been so cautious about branding GitHub as a Microsoft property out the gate. It's trying to ease devs into the idea that the company is something devs can like, and I wouldn't be surprised if this psychological strategy is at work with the npm acquisition, too.
>I once saw some Microsoft guys demo something cool at a conference, and they had to basically apologize that they were from Microsoft
Microsoft, "the people" were never bad. They did a lot of cool things, they had great coders and scientists. The upper management did bad things, and made poor decisions, guys like Ballmer and business strategists.
With the new management things changed a lot and I hope Microsoft will keep up at doing good things.
Not all things are rosy: desktop development with MS tools suck, they changed framework after framework, Windows Forms and WPF aren't cross platform, C++ desktop programming is still Windows only, and not Visual at all. I don't get why the naming: "Visual C++" since QT or C+= Builder are much more "Visual" than MFC and Winapi.
But I guess desktop doesn't bring MS too much money so they don't care enough about it. If that's the case, I don't get why they don't promote and support a third party, cross platform development framework like Uno or Avalonia.
> capture a tool/ecosystem that developers liked using and benefit from that positive sentiment
So...appeal to devs, something something, money??
I really like how MS are improving our tools and embracing open source, I really do. But I've never quite understood how the return on investment in these things justify the cost. I just struggle to the an obv big picture here.
I.e. is it incorrect to think of the GH acquisition as mostly an azure marketing expense?
I have a limited understanding of this but I think the goal is to have a complete developer ecosystem. When most software was deployed to Windows desktops, they owned the developer environment - Visual Studio.
Now most software is written and deployed on the cloud, but not their cloud. But they could make the dev experience compelling and easy - write your code in VS Code, which automaticallgy integrates with Github. Github is mostly free until you're a large company so why not use it. Of course you need to run CI and Github makes that easy so go ahead and add a single file to configure that.
Now you have a build artifact ready to deploy on github. Would you like to click a single button and have that deployed to Azure? They'll also throw in monitoring if you do it. Azure bills is the pot of gold at the end of the developer experience rainbow.
But this is just speculation. I don't understand business very well.
They can, and will have to continue to, thank MS legal for that bad rep..
While Upper Management has lessened some of their excesses, they still tend to be very aggressive in some area's, and time will tell which 1/2 of the company will win. The Open Collaboration group, or the "We want to control the world" group, there is still an internal struggle there
Many believe the "We love Open Collaboration" is just a facade and the "real Microsoft" will revel itself in a few years
I agree with this take. I'm primarily JavaScript/TypeScript developer, doing both front end and back end web app development. Microsoft being a good steward of TypeScript (and now the js ecosystem) inclines me to give C# and .NET Core a shot, and probably Azure down the line.
I for one appreciate that Microsoft is allowing these dev related acquisitions, such as Github, to maintain an individual brand image.
I can't think of anything worst if they decided to reskin/theme these brands to be all Microsoft-y.
Accruing positive sentiment in the dev community through acquisitions of someone else's nice things. Who came up with that crazy idea? GitHub isn't Microsoft. You can't buy brand loyalty.
Ah, but they can. The older folks will remember GitHub being a separate thing but as hundreds of thousands of new devs enter the ecosystem every year, they are introduced to a cool GitHub tool by Microsoft. They don’t know that it was separate nor the animosity towards MS.
You can’t change people’s minds on emotional baggage like this. You just wait for them to die off/retire and target the new blood.
Personally I've always found GitLab's design much more user friendly and the code and repository layout is much nicer than the spartan and cold GitHub.
About all Microsoft has dared to change (at least transparently) since obtaining GitHub has been the pricing page, and objectively speaking right now... it's a hot confusing mess.
A mascot featuring tentacles is all too fitting here. I don't know a lot about NPM or JS but it's being described as a dumpster fire which is also all too fitting. And I'm able smirk as I write that with no animosity or skin in the game.
You don't think google has telemetry on every file you open in google docs? Same problem - different ecosystem. In fact, you literally have to ask google for your file in order to view it in the first place.
Obviously yes, Google is scanning your Gmail emails and doing telemetry on every document you put into their services. That gets a decent amount of attention but it's no where near the amount of information Microsoft gets from you if you happen to be a Windows user. Google's search history on you would pale in comparison in terms of profiling, etc.
That's a pretty awful article from the age of "putting windows 10 and spying in the title generates easy clicks".
The quoted statement had nothing to do with Windows 10 and was specifically do with Microsoft's digital services like Hotmail & OneDrive, which the statement has now been updated to clarify:
"Finally, we will retain, access, transfer, disclose, and preserve personal data, including your content (such as the content of your emails in Outlook.com, or files in private folders on OneDrive), when we have a good faith belief that doing so is necessary to do any of the following:"
Microsoft doesn't care about your local files, but you agree with the generic "to comply with laws and protect our systems" statement if you decide to use their cloud services.
I think I posted that pretty ironically and with enough glib not to be taken too seriously. Oh well. To say their not profiling users using their OS is insane. Private repositories can't even be encrypted on GitHub. What another absolutely insane joke.
Getting to decide what goes into npm (the client tool) and what does not allows them to focus the ecosystem onto brands that they own and encourages people to buy their proprietary software and services.
It also provides them the option of cutting off third-party tools (e.g. yarn) in the future if they deem it beneficial.
My prediction is that they will prioritize the platform features that can only be accessed by first-party, branded tools, like the forthcoming GitHub mobile app. Eventually they will stop maintaining support for the APIs that the other tools use, and it'll be Microsoft tools from end-to-end. Pushing to GitHub and publishing to NPM from right within VS Code, et c.
Of course, using them on Windows will always work best, and deploying to Azure will always be easiest.
I’m less pessimistic. Microsoft has engineers with a business mindset in charge. For now, it’s true. But we saw node vs IO.js — Microsoft can’t afford a fork or they lose their own control. They won’t close open source, they’ll add subscription or cloud usage fees for large enterprises. The value, like GNU “open core” is that open is popular, while enterprise features on the other hand are expensive. The enterprise is being led now by open source because it’s cheap, then value add to solve the problems they still have around control and oversight, etc.
If anything, I expect basic Visual Studio internals will eventually get open sourced as cheaper to maintain that way than Roslyn rebuild-all-the-things. And VS Code will adopt them and continue to cannibalize VS mindshare.
The first word that came to mind was "integration", but I suppose "control" is another way of putting it.
I doubt Microsoft would intentionally degrade developer experience, like "cutting off third-party tools". Rather, they're seeking to gain market advantage from the tight integration of services. It would make sense for them to encourage third-party tools to play well in that "Microsoft ecosystem".
> Pushing to GitHub and publishing to NPM from right within VS Code
That's exactly what I picture coming soon, if not here already. Also: develop in VS Code, click to build, push to GitHub, deploy to Azure.
It's worth noting that TypeScript is a Microsoft product. As TypeScript has become increasingly popular, I'd imagine that Microsoft has also paid increasing amounts of attention to the JavaScript ecosystem.
Looking at the massive growth in frontend technologies within the last decade, it’s easy to see why Microsoft wants to be a little ahead of the curve this time.
Synergy. End to end tracing of dependencies. Licensing mirrors subsets to on premise clients. Single point of flow for code to published package. There’s plenty of places to extend money they’re making elsewhere.
Lowering the entry barrier for JS developers increases the plausible TAM if you make money from downstream developer tools/services, described as 'commoditizing your complement`. (ref https://www.joelonsoftware.com/2002/06/12/strategy-letter-v/)
Maybe to be able to be "legally compelled" to distribute backdoors onto linux servers / developer machines running npm in exchange for being awarded the JEDI contract.
Money prevents it. It takes money to host things and pay people to work on infrastructure. While people often volunteer to contribute to OSS products because they like or use them, not many are willing to write infrastructure that can handle this kind of traffic in their spare time. Even if you can find someone to donate the time, you'd still need to fund that infra in some way. Having an infra company (say, Google donates a bunch of GCP credits) to cover the hosting costs still puts the project at risk if the host company decides to stop funding.
Whatever happened to people hosting things in an old computer in their basement? That used to be a more popular thing back in the days before the cloud came about and before we had these stable broadband connections. Obviously an infra like npm couldn't deliver with such a setup but at scale, who knows
Was that ever really a thing though? When I dig through my memory (and READMEs on old hard drives) I see a lot of .edu addresses. Seems the good-old-days of the internet wasn't about hosting things on an old computer in your basement but rather hosting things on an old computer in your school's basement.
And around the time when home connectivity became good enough that people considered home hosting was also around the time Slashdot was created.
Maybe you don't remember BBS systems. I know a bunch of people who hosted BBS systems "in their basement", hooking up multiple phone lines (4 or more) and multiple modems connected to a C64, or an Amiga, or a PC. And yes, we were calling all over the country (and the world) to exchange software, programming tools, games, etc. The good old days.
Start a non-profit 501(c)(3). Only a non-profit can acquire the assets of another non-profit, so it's somewhat of a poison pill. Budgets for orgs listed below are anywhere between $100k up to a few million dollars per year. This doesn't mean you can't run on a shoe string; Hacker News and Pinboard run on single servers with hot spares.
Examples: Let's Encrypt (Certs), Internet Archive (Culture), Quad9 (DNS), Wikipedia (Knowledge), OpenStreetMap (GIS), Python Packaging Authority ["PyPi"] (as part of the Python Software Foundation)
EDIT: Seriously, start non-profits whenever considering implementing technology infrastructure you're unlikely to want to extract a profit from and are seeking long term oversight and governance.
There are laws, non-profits executives & boards must follow, but there is nothing stopping any of the non-profits you listed from selling any assets you listed with them as long as doing so is in the best interest of the nonprofit; which at best is a subjective topic, at worst, easily dismissed given properly crafted pretext & reasoning.
Generally speaking, much like for profits, it is the people that run it that decided its culture, not a legal structure.
Legal structures don't enforce good behavior, but it always helps to nudge folks in the right direction. As you mention, culture comes from people, and culture is the foundation of every org.
There's nothing stopping someone from pulling code from an alternate package registry, or directly from someone's computer. So all the required decentralization infrastructure already exists. For people to use it, though, it has to be convenient.
The average developer isn't interested in showcasing their social/political views, starting a revolution or building the future of the internet. They just want to get the job done as quickly and effectively as possible and go home.
Are you interested in mitigating risk by reducing central dependencies? I am. But it's too costly right now.
All being equal, though, we should try to avoid technical systems with unnecessary trust relationships and critical concentrated dependencies, though.
And if we must have a concentrated dependency, we should do our best to pick very trustworthy ones-- both based on their track record and their likely future interests relating to organizational structures.
If you don't trust the code itself – it doesn't matter where it is hosted. Validation/audits etc. are essential in any case.
If you don't trust the reliability/uptime of the central service — it's trivial to create a private mirror containing the packages you want. Every internal build system I have seen does this already.
All kinds of risks. Risks of being co-opted. Risks of no longer being maintained. Risks of poor administration. The risks you mention-- availability, etc.
NPM's had its share of catastrophes and controversies that have made things tougher and caused harm to downstream users. It has also saved lots of effort.
Focusing on e.g. mirroring misses the point, IMO: One's dependency on something fundamental like package infrastructure isn't to survive one deployment or minimum sustaining, but to be an ongoing part of your technology stack. Yes, you can move on, but it'll be costly. If the component decides to start to suck, you're going to feel pain.
I remember some years ago when npm was having a lot of stability issues because the guy just was not being given adequate time/resources from Joyent or whatever, I made a post on r/node saying now was the time for a fully decentralized package registry.
The post if I remember was mostly ignored, but received a few downvotes and maybe a couple of negative comments.
Based on that, it seems that what's preventing decentralization from taking off is ignorance and apathy.
A day or two later the guy announced npm, Inc. if I remember.
There are actually a lot more developers that have accepted a federated services worldview than a peer-based fully distributed one. But there are package registry projects along both of those lines.
From my own attempts at developing a decentralized service I've learned that there is no free lunch. Consensus is still unsolved without resorting to a nuclear option like a blockchain. Federation is a far better strategy. Let people host their own instances and let instances interact with each other.
NPM by it's very nature is a centralized repository, not really an agent of decentralization, where you'd get code from the authors' sites/servers directly.
Ubuntu and Debian have mirrors of their apt repositories hosted all around the world by groups, schools, businesses, etc. There is no single point of failure, you grab from the best mirror near you.
That still isn't decentralization: There may be a bunch of mirrors, but there is still a single "source of truth". One entity determines what is acceptable and valid in a repository model, as the entire ability to trust packages you install tends to be on the assumption that entity is making good choices.
discoverability mostly... the social aspect a second, smaller issue.
I think this is probably a good thing in general, and should maybe lead to some interesting enhancements, and maybe even finally solve the distribution of binary modules at a better level.
Question from a new JS developer: Should I be using NPM to manage my dependencies?
I have recently started getting into JS programming. I have thus far avoided NPM, because I've been trying to use CDNs for all my external dependencies.
My thinking is that it saves me bandwidth costs and potentially saves my user's bandwidth as well if they get a cache hit.
I get the downsides are that I don't control the CDN and they could go offline, but honestly I expect I am much more likely to go down from some mistake in my own deployment rather than a well known CDN being offline.
I am wondering if I am missing something though, because absolutely every JS package I read about suggests you use NPM (some also link a CDN, many don't). Should I be using NPM to manage my JS dependencies instead of using CDNs?
IIRC it turns out the cache hits from CDN'ed Javascript files ended up being fairly low and neglible, due to how many different versions there are of everything. Better just reduce the file size.
I feel like bootstrap and jquery stand a decent chance of being caches in a large enough portion of the user base.
And even ignoring my user's bandwidth, it would still save me significant bandwidth (depending on the size of my website).
I guess eventually your site might grow such that your dependencies are not a significant portion of your total download size, but I am not currently there.
Do you 'feel like' or do you actually know? It's important to make decisions based on actual data. I haven't researched it in depth, but if these asset sizes matter to you, then it is worth researching.
If you can afford to load an image, you can afford to send a JS file. Compile all your JS in to one file and host it yourself. The original sell for JS CDNs was that the library would already be cached from the user visiting another site, and that it would serve it from a local edge. It's really not that big of a sell, and comes with a bunch of risk.
CDNs are far less reliable than my own site, and if my own site is down it's not much help that the CDN is up. Pull in two libraries from CDNs and suddenly you have three points of failure instead of one. Their traffic spikes become your traffic spikes, their downtime is your downtime. And for what? The possibility that maybe the user had that one tiny js library cached, or that the CDN has a node 100ms closer? Not worth it.
Personally, I only use external library CDNs during early stages of development, or quick prototypes.
There are advantages to having all needed assets locally. The main point for me is to minimize external dependencies during runtime - fewer points of failure. Also, vendor libraries can be a single minified bundle served from the same ___domain. In production they can be moved to a CDN, i.e., CloudFlare.
Using NPM makes sense once you start having more than a few dependencies, or a build step.
On the other hand, if you can get by with library CDNs and don't feel the need for NPM - I'd say that sounds fine, to keep it simple and practical.
He didn't say which developers though. Arguably, Office and enterprise developers have been served fairly well even during his tenure. As long as you colored between the lines, the experience was alright.
I hope this only goes as far as being able to sign up with and link a GitHub account to NPM. Any tighter integration seems like it would be in bad faith, in terms of allowing integration with other git services/non-GH package hosting.
On the contrary, I'd vasty prefer if a package repository required the source be hosted within the same account (even if just a mirror) that is offering the package, so that they can verify the authenticity/reproducibility of anything inside.
He did mention that they were using their enterprise customers to subsidize the cost to allow them to offer Teams for free, so maybe if they get enough enterprise customers using GitHub Packages we might see unlimited free private packages for individuals/teams. I don't see a ton of value for GitHub/MS in that situation, but maybe.
I feel like Microsoft has actually built a sustainable, developer-friendly, OSS-friendly business around Azure, Visual Studio, and GitHub.
I could imagine other potential acquirers just wanting to do an acquihire, and deprecating the service a few months or years later (when the technical debt is too high to continue running the code in maintenance mode).
Why should anyone? Open source has always been about freedom. MS hasn't acquired the tools to my knowledge, they've acquired popular infrastructure built around tools. Anyone can still start GutBub based on the same `git` source tomorrow if they wanted.
To be sure a lot of this may be about talent as well. They may need to be liked in the community otherwise they have a hard time hiring top talent. They are buying a community since they probably couldn't build it themselves.
Because open source has always been defined in its very definition by its definition to the for-profit software industry, which Microsoft basically invented? Because in spite of having made a turn towards open source, Microsoft once treated it like a mortal enemy, akin to cancer? Because of how other large companies like Google have smothered open source projects?
I'm personally not worried about the acquisition (I think it's a net positive for the community), but there are plenty of reasons to discuss this interaction.
Even people change, why do you find weird that companies change?
>Because in spite of having made a turn towards open source, Microsoft once treated it like a mortal enemy, akin to cancer?
MS was in the business of selling operating systems and desktop software. Not only they realized that open source doesn't threaten that territory (remember year of the Linux desktop?), but they've changed the revenue model and are making much more money now from selling services than by selling software.
>Because of how other large companies like Google have smothered open source projects?
Why should we judge what one company does based on what other company does?
I am sure no company does something for "good of humanity" but to earn money. What matters is if in the process of making money, they also do good things or bad things.
Microsoft has stopped doing bad things and started doing good things. I only hope more companies will follow.
Parent was saying "why discuss this at all?" and I'm saying here are some reasons! And you're doubling down while... having the discussion. Case closed. :)
It's a common misconception that open source is antithetical to profit. The concern is really about the freedom to inspect, modify and distribute the codes which run your computer without restriction.
I hear you, I think MS is prone to mismanage the nicest things we have. And yes part of this will be some PR mess up.
We definitely cannot trust a large cash reserve to guarantee the survival of these technologies, it's just not how businesses work.
#notallopensource, but the Free Software movement part of open source, for sure, you know? Okay I've re-read my original comment, you're right, it was too encompassing.
Yep. IMHO if Google would want to make a move here, now would be a good time. And as the creators of Kubernetes, they have some nice incentive to try to stay relevant.
MS gobbling this one up would not be a big surprise.
No. Unfortunately, there is little we can do. Most people don't take a strategic or ethical view to their development stack: they are getting a job done, so they take the most convenient / fastest / easiest way to do so (price being equal). And if they get screwed 5 years from now on, who cares - their employer will pay them again to re-do the work.
I really hope so, there is probably a bunch of malware on there that people don't even know about when is the last time that place has been audited? I don't blame them they are probably running with limited funds etc...
The acquisition doesn't mean all existing NPM packages are suddenly going to get dumped into GitHub repos, though. The services still do two different things.
