Since every 3rd message on this thread (at the time I wrote this) is about how Google underpaid for this bug, some quick basic things about vulnerability valuations:
* Valuations for server-side vulnerabilities are low, because vendors don't compete for them. There is effectively no grey market for a server-side vulnerability. It is difficult for a third party to put a price on a bug that Google can kill instantaneously, that has effectively no half-life once discovered, and whose exploitation will generate reliable telemetry from the target.
* Similarly, bugs like full-chain Android/Chrome go for hundreds of thousands of dollars because Google competes with a well-established grey market; a firm can take that bug and sell it to potentially 6 different agencies at a single European country.
* Even then, bounty vs. grey market is an apples-oranges comparison. Google will pay substantially less than the grey market, because Google doesn't need a reliable exploit (just proof that one can be written) and doesn't need to pay maintenance. The rest of the market will pay a total amount that is heavily tranched and subject to risk; Google can offer a lump-sum payment which is attractive even if discounted.
* Threat actors buy vulnerabilities that fit into existing business processes. They do not, as a general rule, speculate on all the cool things they might do with some new kind of vulnerability and all the ways they might make money with it. Collecting payment information? Racking up thousands of machines for a botnet? Existing business processes. Unmasking Google accounts? Could there be a business there? Sure, maybe. Is there one already? Presumably no.
A bounty payout is not generally a referendum on how clever or exciting a bug is. Here, it kind of is, though, because $10,000 feels extraordinarily high for a server-side web bug.
For people who make their nut finding these kinds of bugs, the business strategy is to get good at finding lots of them. It's not like iOS exploit development, where you might sink months into a single reliable exploit.
This is closer to the kind of vulnerability research I've done recently in my career than a lot of other vuln work, so I'm reasonably confident. But there are people on HN who actually full-time do this kind of bounty work, and I'd be thrilled to be corrected by any of them.
Isn't this reconstruction a bit on the slim side? Aquinas was reportedly, let's say, a man of portly presence.
I can't find a scholarly source on the matter, at the moment, but here are two quotes I found on the website of a nun[1] (no less, so probably written in good faith):
> St. Thomas was a huge heavy bull of a man, fat and slow and quiet; very mild and magnanimous but not very sociable; shy, even apart from the humility of holiness; and abstracted, even apart from his occasional and carefully concealed experiences of trance or ecstasy. (G.K. Chesterton)
> St. Thomas Aquinas was a compulsive over-eater who was not just fat but morbidly obese and physically grotesque. (Myron Shibley)
(Fun fact, there's a reference to this in Umberto Eco's The Name of the Rose, alluding to difficulties with the transport of the body over a staircase, which coincides with circumstances of G.K. Chesterton's passing, as described on that page.)
Complete hardware + software setup for running Deepseek-R1 locally. The actual model, no distillations, and Q8 quantization for full quality. Total cost, $6,000. All download and part links below:
Motherboard: Gigabyte MZ73-LM0 or MZ73-LM1. We want 2 EPYC sockets to get a massive 24 channels of DDR5 RAM to max out that memory size and bandwidth. https://t.co/GCYsoYaKvZ
CPU: 2x any AMD EPYC 9004 or 9005 CPU. LLM generation is bottlenecked by memory bandwidth, so you don't need a top-end one. Get the 9115 or even the 9015 if you really want to cut costs https://t.co/TkbfSFBioq
RAM: This is the big one. We are going to need 768GB (to fit the model) across 24 RAM channels (to get the bandwidth to run it fast enough). That means 24 x 32GB DDR5-RDIMM modules. Example kits: https://t.co/pJDnjxnfjghttps://t.co/ULXQen6TEc
Case: You can fit this in a standard tower case, but make sure it has screw mounts for a full server motherboard, which most consumer cases won't. The Enthoo Pro 2 Server will take this motherboard: https://t.co/m1KoTor49h
PSU: The power use of this system is surprisingly low! (<400W) However, you will need lots of CPU power cables for 2 EPYC CPUs. The Corsair HX1000i has enough, but you might be able to find a cheaper option: https://t.co/y6ug3LKd2k
Heatsink: This is a tricky bit. AMD EPYC is socket SP5, and most heatsinks for SP5 assume you have a 2U/4U server blade, which we don't for this build. You probably have to go to Ebay/Aliexpress for this. I can vouch for this one: https://t.co/51cUykOuWG
And if you find the fans that come with that heatsink noisy, replacing with 1 or 2 of these per heatsink instead will be efficient and whisper-quiet: https://t.co/CaEwtoxRZj
And finally, the SSD: Any 1TB or larger SSD that can fit R1 is fine. I recommend NVMe, just because you'll have to copy 700GB into RAM when you start the model, lol. No link here, if you got this far I assume you can find one yourself!
And that's your system! Put it all together and throw Linux on it. Also, an important tip: Go into the BIOS and set the number of NUMA groups to 0. This will ensure that every layer of the model is interleaved across all RAM chips, doubling our throughput. Don't forget!
Next, the model. Time to download 700 gigabytes of weights from @huggingface! Grab every file in the Q8_0 folder here: https://t.co/9ni1Miw73O
Believe it or not, you're almost done. There are more elegant ways to set it up, but for a quick demo, just do this. llama-cli -m ./DeepSeek-R1.Q8_0-00001-of-00015.gguf --temp 0.6 -no-cnv -c 16384 -p "<｜User｜>How many Rs are there in strawberry?<｜Assistant｜>"
If all goes well, you should witness a short load period followed by the stream of consciousness as a state-of-the-art local LLM begins to ponder your question:
And once it passes that test, just use llama-server to host the model and pass requests in from your other software. You now have frontier-level intelligence hosted entirely on your local machine, all open-source and free to use!
And if you got this far: Yes, there's no GPU in this build! If you want to host on GPU for faster generation speed, you can! You'll just lose a lot of quality from quantization, or if you want Q8 you'll need >700GB of GPU memory, which will probably cost $100k+
(Actually the very first one, in that history, was an intercontinental cruise missile—a jet weapon that slightly predated (~1958) rockets powerful enough to cross oceans. ICBM's came a bit later. I'm pretty sure the first generation were pure-analog circuits, but I forgot where I read about that).
Over the holidays, we published a post[1] on using high-precision few-shot examples to get `gpt-4o-mini` to perform similar to `gpt-4o`. I just re-ran that same experiment, but swapped out `gpt-4o-mini` with `phi-4`.
`phi-4` really blew me away in terms of learning from few-shots. It measured as being 97% consistent with `gpt-4o` when using high-precision few-shots! Without the few-shots, it was only 37%. That's a huge improvement!
By contrast, with few-shots it performs as well as `gpt-4o-mini` (though `gpt-4o-mini`'s baseline without few-shots was 59% – quite a bit higher than `phi-4`'s).
Note also that the modern "UTC epoch" is January 1, 1972. Before this date, UTC used a different second than TAI: [1]
> As an intermediate step at the end of 1971, there was a final irregular jump of exactly 0.107758 TAI seconds, making the total of all the small time steps and frequency shifts in UTC or TAI during 1958–1971 exactly ten seconds, so that 1 January 1972 00:00:00 UTC was 1 January 1972 00:00:10 TAI exactly, and a whole number of seconds thereafter. At the same time, the tick rate of UTC was changed to exactly match TAI. UTC also started to track UT1 rather than UT2.
So Unix times in the years 1970 and 1971 do not actually match UTC times from that period. [2]
The most important operation in QNX is MsgSend, which works like an interprocess subroutine call. It sends a byte array to another process and waits for a byte array reply and a status code. All I/O and network requests do a MsgSend. The C/C++ libraries handle that and simulate POSIX semantics. The design of the OS is optimized to make MsgSend fast.
A MsgSend is to another service process, hopefully waiting on a MsgReceive. For the case where the service process is idle, waiting on a MsgReceive, there is a fast path where the sending thread is blocked, the receiving thread is unblocked, and control is immediately transferred without a trip through the scheduler. The receiving process inherits the sender's priority and CPU quantum. When the service process does a MsgReply, control is transferred back in a similar way.
This fast path offers some big advantages. There's no scheduling delay; the control transfer happens immediately, almost like a coroutine. There's no CPU switch, so the data that's being sent is in the cache the service process will need. This minimizes the penalty for data copying; the message being copied is usually in the highest level cache.
Inheriting the sender's priority avoids priority inversions, where a high-priority process calls a lower-priority one and stalls. QNX is a real-time system, and priorities are taken very seriously. MsgSend/Receive is priority based; higher priorities preempt lower ones. This gives QNX the unusual property that file system and network access are also priority based. I've run hard real time programs while doing compiles and web browsing on the same machine. The real-time code wasn't slowed by that. (Sadly, with the latest release, QNX is discontinuing support for self-hosted development. QNX is mostly being used for auto dashboards and mobile devices now, so everybody is cross-developing. The IDE is Eclipse, by the way.)
Inheriting the sender's CPU quantum (time left before another task at the same priority gets to run) means that calling a server neither puts you at the end of the line for CPU nor puts you at the head of the line. It's just like a subroutine call for scheduling purposes.
MsgReceive returns an ID for replying to the message; that's used in the MsgReply. So one server can serve many clients. You can have multiple threads in MsgReceive/process/MsgReply loops, so you can have multiple servers running in parallel for concurrency.
This isn't that hard to implement. It's not a secret; it's in the QNX documentation. But few OSs work that way. Most OSs (Linux-___domain messaging, System V messaging) have unidirectional message passing, so when the caller sends, the receiver is unblocked, and the sender continues to run. The sender then typically reads from a channel for a reply, which blocks it. This approach means several trips through the CPU scheduler and behaves badly under heavy CPU load. Most of those systems don't support the many-one or many-many case.
Somebody really should write a microkernel like this in Rust. The actual QNX kernel occupies only about 60K bytes on an IA-32 machine, plus a process called "proc" which does various privileged functions but runs as a user process. So it's not a huge job.
All drivers are user processes. There is no such thing as a kernel driver in QNX. Boot images can contain user processes to be started at boot time, which is how initial drivers get loaded. Almost everything is an optional component, including the file system. Code is ROMable, and for small embedded devices, all the code may be in ROM. On the other hand, QNX can be configured as a web server or a desktop system, although this is rarely done.
There's no paging or swapping. This is real-time, and there may not even be a disk. (Paging can be supported within a process, and that's done for gcc, but not much else.) This makes for a nicely responsive desktop system.
Unless you grind leetcode problems, it is unlikely one could solve them in 30 minutes (you are supposed to cheat). For example, see whether you can come up with Manacher's algorithm (or any other O(n) solution in 30 minutes https://en.wikipedia.org/wiki/Longest_palindromic_substring#... (the example is from the real interview for senior position (implies multiyear programming experience)). I could see a dynamic programming solution (quadratic) be implemented in 30 minutes but not O(n). Try it for yourself.
bash -- DSL for invoking subprocesses :: what you rather see in a code review: one-line shell pipeline or equivalent Python code? https://stackoverflow.com/questions/295459/how-do-i-use-subp...
sh-like DSL on top of Python literal strings that doesn't invoke shell might be interesting (pep may help)
regex -- DSL for search/replace in text. Useful in moderation
jq -- DSL for search/replace in json. Useful on the command-line
xpath -- DSL for searching trees (hierarchy)
sql/xslt -- I would rather read Python most of the time instead but sometimes it is helpful to have an option of writing SQL directly (pep may help)
toml/json -- writing Python directly is preferable (in the context of .py file)
markdown -- DSL for writing markup (instead of html/word). I wouldn't say no to inline docstring rendering in Python code (pep may help).
The same for (subset of) latex -- for math formulas `$e^{i \pi} -1 = 0$`
dot/plantuml (ebnf, sequence diagrams, etc) could be useful for literate programming-heavy style.
If we consider science as a way to document reality (among other things) then "information -> decision" is an elegant point of view (if information doesn't help make decision, it is just noise). https://news.ycombinator.com/item?id=41881872
It reminds me of "brain (mostly) for movement " (no decisions to make, no brain is necessary). Sea squirts being a prime example: "As an adult, the sea squirt attaches itself to a stationary object and then digests most of its own brain." (Rodolfo R. Llinas
I of the Vortex: From Neurons to Self) https://www.amazon.com/Vortex-Neurons-Rodolfo-R-Llinas/dp/02...
Your model of the world is not perfect so instead of trying to find a globally optimal solution, you are satisfied with a local optimum that exceeds some threshold that has to suffices.https://en.wikipedia.org/wiki/Satisficing
The 20% rest – sidebar for documentation – are not implemented for a reason.
The sidebar to show the documentation is just a presentation detail. What I’d really like as a developer is (a) the ability to attach documentation with a specific scope ranging from small things like a function parameter or variable, though larger things like a function or a set of related functions, up to whole files or packages, and then (b) editors that recognise the scoped documentation and present it contextually.
I’d like all relevant comments to be easily visible when I might want to refer to them. Also, I’d like to know what documentation exists in the first place, which is not always obvious when we rely on things like big banner comments at the top of files/sections or having separate README files scattered through the tree.
> That doesn't really matter for discussing how correct code _should_ be written.
It absolutely does when you're talking about the semantics of virtually every program on earth
> It's equivalent to "if you call malloc(), you should call free()" -- you shouldn't demand that functions you invoke will call free() on your pointer. Same for open files.
There are many cases where the one calling malloc cannot be the one calling free and must explicitly document to callers/callees who is responsible for memory deallocation. This is a good example of where no convention exists and it's contextual.
But open files aren't memory and one cannot rely on file descriptors being closed without errors in practice, so people don't, and you can't just repave decades of infrastructure for no benefit out of ideological purity.
Prisons and psychiatric care provide compelling examples of the boundaries of for-profit models and why they might not always be the best solutions in an area. You can always argue — reasonably I think — that these cases represent anomalous bad apples but I think as long as that incentive structure is there it will pull for this kind of thing at some level, even it's not quite as extreme as this example. I think that profit incentives and lack of competition tend to distort healthcare across all sorts of specialties in much more subtle ways than what was going on at this hospital chain. When threats to personal autonomy start to become the consequence it seems to clarify some of the problems involved, but the same processes are at work in all sorts of areas.
I was surprised to find that it is possible (though unlikely, as it has to happen in a very narrow window of time between two particular bytecodes) for a KeyboardInterrupt to cause a with block to exit without the __exit__ method of the context manager running.
Crowdstrike did this to our production linux fleet back on April 19th, and I've been dying to rant about it.
The short version was: we're a civic tech lab, so we have a bunch of different production websites made at different times on different infrastructure. We run Crowdstrike provided by our enterprise. Crowdstrike pushed an update on a Friday evening that was incompatible with up-to-date Debian stable. So we patched Debian as usual, everything was fine for a week, and then all of our servers across multiple websites and cloud hosts simultaneously hard crashed and refused to boot.
When we connected one of the disks to a new machine and checked the logs, Crowdstrike looked like a culprit, so we manually deleted it, the machine booted, tried reinstalling it and the machine immediately crashes again. OK, let's file a support ticket and get an engineer on the line.
Crowdstrike took a day to respond, and then asked for a bunch more proof (beyond the above) that it was their fault. They acknowledged the bug a day later, and weeks later had a root cause analysis that they didn't cover our scenario (Debian stable running version n-1, I think, which is a supported configuration) in their test matrix. In our own post mortem there was no real ability to prevent the same thing from happening again -- "we push software to your machines any time we want, whether or not it's urgent, without testing it" seems to be core to the model, particularly if you're a small IT part of a large enterprise. What they're selling to the enterprise is exactly that they'll do that.
Containers are offered block storage by creating a loopback device with a backing file on the kubelet’s file system. We noticed that on some very heavily utilized nodes that iowait was using 60% of all the available cores on the node.
I first confirmed that nvme drives were healthy according to SMART, I then worked up the stack and used BCC tools to look at block io latency. Block io latency was quite low for the NVME drives (microseconds) but was hundreds of milliseconds for the loopback block devices.
This lead me to believe that something was wrong with the loopback devices and not the underlying NVMEs. I used cachestat/cachetop and found that the page cache miss rate was very high and that we were thrashing the page cache constantly paging in and out data. From there I inspected the loopback devices using losetup and found that direct io was disabled and the sector size of the loopback device did not match the sector size of the backing filesystem.
I modified the loopback devices to use the same sector size as the block size of the underlying file system and enabled direct io. Instantly, the majority of the page cache was freed, iowait went way down, and io throughout went way up.
Without BCC tools I would have never been able to figure this out.
Double caching loopback devices is quite the footgun.
AOL's server code was excellent. It was written by some people that had done similar things for BITNET and had a deep understanding of Unix programming and how to use event loops for scalability.
It was in C, but the process was expected to run for months without crashing or running out of memory; if it had an abnormal exit, you'd (the owner as set in some code thing) get an email with the core backtrace. If it had a memory leak, ops would be on you quickly. Ops was considered a primary driver of requirements to dev, and absolutely everything could be reloaded into a running server without restarting it. There was a TCP control port where a TCL interpreter inside the service was exposed and generally you wrote the (very simple, CS 101 style) TCL commands to manage the server. It was a "No Threads Kernel", scaled to the dozens or hundreds of physical machines communicating over a very well managed network, and most 1 process per core, and 1 core for the OS. The 200
or so unix developers (as we were called) had a common understanding of how the framework worked and if you were just writing app code it was basically impossible to write slow services. We had technical writers that would interview the developers and write books that could be handed to outside developers and lead to a successful integration with no developer time spent.
The NTK was primarily for sending msgs over the network - we had a principle to never write to disk (which were pretty slow in the 1990s), so everything was just a server to get network messages and then send out other messages in reponse and then assemble the replies/timeouts and send back a msg to the caller. All done over persistent connections established by the infrastructure, the applications just registered callbacks for msg type 'X' which would present one with the caller information, the msg as a buffer, and potentially a "user word" which would be a way to keep server state around between calls.
The layering, from the main select loop thru different layers of TCP connection, SSL handling (if used, not 100% was SSL in the 90s), thru persistent link handling, application msg handling, timers, memory allocation, etc. was done with such art that I felt it to be a thing of beauty.
Google's monorepo, and it's not even close - primarily for the tooling:
* Creating a mutable snapshot of the entire codebase takes a second or two.
* Builds are perfectly reproducible, and happen on build clusters. Entire C++ servers with hundreds of thousands of lines of code can be built from scratch in a minute or two tops.
* The build config language is really simple and concise.
* Code search across the entire codebase is instant.
* File history loads in an instant.
* Line-by-line blame loads in a few seconds.
* Nearly all files in supported languages have instant symbol lookup.
* There's a consistent style enforced by a shared culture, auto-linters, and presubmits.
* Shortcuts for deep-linking to a file/version/line make sharing code easy-peasy.
* A ton of presubmit checks ensure uniform code/test quality.
* Code reviews are required, and so is pairing tests with code changes.
All good points. But I'd argue that the single worst part of gRPC is the impenetrability of its ecosystem. And I think that, in turn, is born of complexity. The thing is so packed with features and behaviors and temporal coupling and whatnot that it's difficult to produce a compatible third-party implementation. That means that, in effect, the only true gRPC implementation is the one that Google maintains. Which, in turn, means that the only languages with good enough gRPC support to allow betting your business on it are the ones that Google supports.
And a lot of these features arguably have a poor cost/benefit tradeoff for anyone who isn't trying to solve Google problems. Or they introduce painful constraints such as not being consumable from client-side code in the browser.
I keep wishing for an alternative project that only specifies a simpler, more compatible, easier-to-grok subset of gRPC's feature set. There's almost zero overlap between the features that I love about gRPC, and the features that make it difficult to advocate for adopting it at work.
I agree with the thesis of this article but I actually think the point would be better made if we switch from talking about optimizing to talking about satisficing[1].
Simply put, satisficing is searching for a solution that meets a particular threshold for acceptability, and then stopping. My personal high-level strategy for success is one of continual iterative satisficing. The iterative part means that once I have met an acceptability criterion, I am free to either move on to something else, or raise my bar for acceptability and search again. I never worry about whether a solution is optimal, though, only if it is good enough.
I think that this is what many people are really doing when they say they are "optimizing", but using the term "optimzing" leads to confusion, because satisficing solutions are by definition non-optimal (except by luck), and some people (especially the young, in my experience) seem to feel compelled to actually optimize, leading to unnecessary perfectionism.
ps — Re-reading that wikipedia article reminds me of how often the topics that make the front page as "new" thoughts have been studied and written about in detail by thinkers of the past. Herbert Simon's Bounded Rationality has a lot to say about the toping of the original link.
If you read about agile, standups are supposed to be conducted among the dev team members who are working together on something, and primarily for their benefit, not as a status update to or for someone else. The PM or whoever is considered the "product owner" in scrum could listen in and help unblock issues, or report delays to other people, but not demand more information or change things up that have already been planned. Of course, it actually working this way is rare, although I have seen it.
I haven't really seen this, but a poorly performing product manager or product owner completely breaks the scrum or agile model (to the extent it works at all). They are assumed to basically know the ___domain and know what needs to be built at a high level, and the software requirements originate from them in the scrum model. If they don't know what the requirements should be or how to communicate them or how to collaborate with the engineers on the requirements, it is completely garbage-in-garbage-out. On the other hand, working with a product owner who is a ___domain expert and happy to help define software to solve their needs can be a joy.
> "Our research has shown that what matters when it comes to delivering high-quality software on time and within budget is a robust requirements engineering process and having the psychological safety to discuss and solve problems when they emerge, whilst taking steps to prevent developer burnout."
There is some virtue to your criticism, but it is also not entirely true. There are things you can check to see if a team is doing Agile for real, like, what's the last process change you experimented with to see if it improved your outcomes? What experiment failed and was dispassionately removed from the process because it didn't work out? When your team's workload nature changed (e.g., from new development to maintenance), how did you adapt your processes to match? Do your processes come up from the team and their experience or do they source from above with effectively no feedback?
At that point, if you are doing those things, there is a certain amount of validity to the criticism that if it didn't work, you really weren't doing it right. Either something external jammed you up so you weren't able to adapt and truly follow agile, your team was personally unable to execute on the adaptation for some reason (structure, personality issues, experience levels, being simply too large to be able to be this flexible because large teams simply need more structure), or the task was simply too hard in the first place for some reason (such as "it doesn't matter how agile and smart the team is, you're not getting 4 people to produce a standards-compliant browser that is also an office suite in six months").
Being present at the conference is the base point of conversation.
But if you’re running out of “how to get to know someone” topics, that’s more generally stuff like.. well, my reply converts to nested conference talk outline here, and so:
- invent a question
- about something cool
- cool to you is fine
- as long as it’s relevant
- the conference is relevant
- so are tangents from it
- sometimes the venue too
- or about them
- ask what’s cool to them
- about the conference
- or tangentially about it
- or the venue architecture
- ask for more details
- or be quiet and sip coffee
- quiet isn’t always an end
- it’s restful
- mirror how long your answers are
- the spice must flow
- don’t drown them in it
- add 10% to be extroverted
- subtract 50% to be introverted
- you can be plainly trying here
- you don’t need to disguise it
- being obvious can be encouraging
- being obvious isn’t bad
- subtlety is not mandatory
- tell them you appreciated the chat
- unless you totally hated it
- it costs nothing to be nice
- say it plainly
- it was nice to chat
- i appreciate what you said
- glad to meet you
- have fun at the conference
When I was hiring data scientists for a previous job, my favorite tricky question was "what stack/architecture would you build" with the somewhat detailed requirements of "6 TiB of data" in sight. I was careful not to require overly complicated sums, I simply said it's MAX 6TiB
I patiently listened to all the big query hadoop habla-blabla, even asked questions about the financials (hardware/software/license BOM) and many of them came up with astonishing tens of thousands of dollars yearly.
The winner of course was the guy who understood that 6TiB is what 6 of us in the room could store on our smart phones, or a $199 enterprise HDD (or three of them for redundancy), and it could be loaded (multiple times) to memory as CSV and simply run awk scripts on it.
I am prone to the same fallacy: when I learn how to use a hammer, everything looks like a nail. Yet, not understanding the scale of "real" big data was a no-go in my eyes when hiring.
> I don’t see any path from continuous improvements to the (admittedly impressive) ”machine learning” field that leads to a general AI any more than I can see a path from continuous improvements in horse-breeding that leads to an internal combustion engine.
You can counter it doesn't necessarily need an AGI here but that doesn't change the fact you can't crank this engine harder and expect it to power an airplane.
> You might be surprised to learn that I actually think LLMs have the potential to be not only fun but genuinely useful. “Show me some bullshit that would be typical in this context” can be a genuinely helpful question to have answered, in code and in natural language — for brainstorming, for seeing common conventions in an unfamiliar context, for having something crappy to react to.
> Alas, that does not remotely resemble how people are pitching this technology.
What an excellent resource! (And yes Outlook is a pain and supports so very little!)
We've tried building email templates for notifications for our apps where I work, and it has typically been a pain. We have since swapped to using mjml (https://mjml.io/) to build the templates, and it's working wonders. The output seems the be the most compatible with all different devices that we've tested on.
The other tool we enjoy using is Litmus (https://litmus.com), which allows you to throw in an email template and see what it looks like on all kinds of apps and devices. Other thread here mentions https://testi.at/ as well, which we've also had success with.
All of these have been really invaluable to designing emails for our apps.
* Valuations for server-side vulnerabilities are low, because vendors don't compete for them. There is effectively no grey market for a server-side vulnerability. It is difficult for a third party to put a price on a bug that Google can kill instantaneously, that has effectively no half-life once discovered, and whose exploitation will generate reliable telemetry from the target.
* Similarly, bugs like full-chain Android/Chrome go for hundreds of thousands of dollars because Google competes with a well-established grey market; a firm can take that bug and sell it to potentially 6 different agencies at a single European country.
* Even then, bounty vs. grey market is an apples-oranges comparison. Google will pay substantially less than the grey market, because Google doesn't need a reliable exploit (just proof that one can be written) and doesn't need to pay maintenance. The rest of the market will pay a total amount that is heavily tranched and subject to risk; Google can offer a lump-sum payment which is attractive even if discounted.
* Threat actors buy vulnerabilities that fit into existing business processes. They do not, as a general rule, speculate on all the cool things they might do with some new kind of vulnerability and all the ways they might make money with it. Collecting payment information? Racking up thousands of machines for a botnet? Existing business processes. Unmasking Google accounts? Could there be a business there? Sure, maybe. Is there one already? Presumably no.
A bounty payout is not generally a referendum on how clever or exciting a bug is. Here, it kind of is, though, because $10,000 feels extraordinarily high for a server-side web bug.
For people who make their nut finding these kinds of bugs, the business strategy is to get good at finding lots of them. It's not like iOS exploit development, where you might sink months into a single reliable exploit.
This is closer to the kind of vulnerability research I've done recently in my career than a lot of other vuln work, so I'm reasonably confident. But there are people on HN who actually full-time do this kind of bounty work, and I'd be thrilled to be corrected by any of them.