How is it that Brave managed to build an indexer and remove dependence on Bing in less than two years but DuckDuckGo hasn't been able to do it in a decade.
DDG probably doesn't want to? On the time horizons they're thinking about, it's probably more expensive to develop competitive tech and keep it working than it is to pay for API access.
"Today Brave announced the acquisition of Tailcat, the open search engine developed by the team formerly responsible for the privacy search and browser products at Cliqz, a holding of Hubert Burda Media. Tailcat will become the foundation of Brave Search..."
Seems so. Same references to Hubert Burda Media being the majority holder if I follow links from that mozilla post to the Cliqz Github repo, then back to their website.
The search index is relatively easy, the ad marketplace is hard. DDG is likely hooked on tens or hundreds of millions of dollars in Bing revenue which is a tough habit to kick.
Once you’re hooked on Google or Bing search ads for revenue, they can control everything you are allowed to do within a product via compliance requirements. It’s a clever and effective form of regulatory capture.
After the invasion of Ukraine they announced they would be removing sites "associated with Russian disinformation." They haven't provided a definition of what that includes.
Lots of DDG users were upset because this is the type of thing they objected to with Google.
In DDG's defense, some of the censorship was required by law - we don't know if DDG went further than demanded - and Eich himself will tell you that Brave had to remove some results as well.
No, I never said we removed anything due to some nation state's MiniTrue calling it misinformation or disinformation.
All search engines must comply with laws (CSAM, right to be forgotten or whatever it is called now) in various regions. That compliance is not in any way the same category as what DDG boasted in March 2022 that it was doing.
But DDG did not have to virtue-signal as it did in March 2022 due to force of law or even unlawful threats from a nation-state. Categorically different -- there's no comparison.
Thanks for the heads up, this will certainly affect how (and how often) I use DDG in the future.
I despise disinformation and war propaganda, but I decide for myself what content falls in those categories – I don't need my search engine to decide it for me.
If they censor "misinformation" in one case, what's to stop them from censoring other "misinformation"?
Maybe the COVID lab leak theory is deemed misinformation and removed from your search result. But it's not 100% proven either way, who is to say that this is "misinformation"?
It's one thing to suggest an opinion that's just more popular, it's another to remove the competing opinions
You know that reasoning can be applied to literally any removal right? There's _tons_ of stuff that it would be pretty reasonable to remove (and I'm sure you'd agree that things like CP shouldn't be on there etc.). There's also plenty of stuff that they would likely _have_ to remove if the state told them to (I don't have direct evidence of this happening, so take it as a hypothetical), while this isn't necessarily a _good_ thing, it's also not a slippery slope.
I'd also like to highlight the thing where you implied that Russian state propaganda is just an opinion or a theory (and therefore just as valid as any other innocuous opinion or theory), which is at best intellectually dishonest.
CP and unlawful content is what the state forces them to remove. Other opinions are exactly the slippery slope I am talking about.
Russian propaganda is propaganda and lies, but if you remove one type of lies, then why not COVID misinformation? No government is FORCING them to any of this, so it's a slippery slope
It's a search engine's job to rank results; there is no other way to do it: only one link can be in position #1, only one link can be in position #2, etc.
Or in other words: "editorial guidance" is pretty much the entirety of a search engine's job: you give it a large set of documents (the internet), some user input (what you typed in the search box), and it ranks – or "editorializes" – the set of documents to something useful for you.
And at the same time you also have to account for SEO haxx0rs and outright malicious actors who will try to phish your CC details.
Do you want some crackpot website if you search for "Barrack Obama" which claims that he is literally the anti-Christ to be at #1? Or even on the first page at all? Or rolexxxx.com if you search for "buy rolex"? Or bank-of-amerrrica.ru if you search for "Bank of America"? Probably not. A naïve ranking algorithm will end up with that.
There is no perfect way to do this; it's a hard problem. Platitudes like this make it sound easy, but it's not.
There is a massive difference between ranking stuff based on relevance and not getting RT articles when... searching for RT. (They rolled back the block pretty quickly, so they seem to agree with that too)
You are basically arguing for a slippery slope argument. Because they already need some editorial control to filter spam and obviously irrelevant material does not mean that every type of filtering/block listing is ok.
I personally totally get how it can be offputting to people if a search engine starts hiding websites while openly saying that they do it for a political reason. Downranking would be fine, but blocking a news source (as bad as RT is at being that) that isn't spammy or playing with SEO is just different.
Yes, I know, everything is political and all. But that's the point! Blocking RT was obviously more so about politics than filtering fake news or trash results.
It pretty much was just about fake news and trash results, actually. RT really is pure garbage, by its very intent and purpose.
I do agree that blocking RT outright was overreach and somewhat silly. However, I imagine it comes more from a gut feeling of disgust and "this is bullshit" then than some ultraelitist, "gotta protect the unwashed masses from harmful ideas that may bring the global order into question" motive that people in this thread seem to want DDG to have.
Yes, but they could've just blacklisted or downranked it based on already existing rules about fake news. But they instead went ahead and basically created a rule for Russian content specifically.
I don't even doubt DDGs intentions, but that's the thing... in most cases, people who want to suppress or filter out speech/news/etc usually have good intentions because as you said they genuinely think it is harmful content. That's the whole issue! And it's why it's usually such a thorny debate.
All of the examples you mentioned involved irrelevant search results (as in, I have never met a soul who wants spam & scams), whereas GP is implying he wants relevant ones.
To flip your example, if I wanted to search for "Barack Obama Anti-Christ," I'd want to get those results back, and not have DDG or whatever other authority decide categorically that I shouldn't see them.
"Editorial guidance" is a wide spectrum from favoring what the editor thinks is most relevant for the reader to the editor actively censoring content the reader wants to see for external reasons. To be charitable, I read GP as objecting to the latter and not the former.
To call what DDG is doing (or what a newspaper does when it chooses not to print your foaming, incoherent editor to the letter -- as is, you know, its right) "censorship" is just silly.
I don't think what you want exists or has ever existed. A search engine that does not exercise judgement about relevance and quality will just return noise.
> but ultimately it's a matter of editorial control, not censorship.
So is all censorship.
No censor calls censorship "Censorship". An example from my country of birth:
Main Office of Control over the Press, Publications and Performances, since 1981 the Main Office of Control over Publications and Performances - the central office of state censorship in the Polish People's Republic. It was a censorship body (analogous institutions were present in all countries of the so-called Eastern Bloc) examining all forms of official information communication from the perspective of their compliance with the current state policy, and prohibiting the dissemination of unwanted information and content by the ruling communist party.
The name "The Censors" was only adopted after the collapse of communism.
I rub my eyes in amazement every time I read people on HN praising censorship and rejoicing that someone will decide for them what they can read and what they can't.
I am not able to understand how foolish one has to be to not realize that eventually the censorship organs will be used against you too.
Perhaps you think you will always hold the "correct" beliefs in which case I admire your lack of imagination.
I miss "hackers" from the 90s with some actual backbone.
I live in a country that's heading down that path. They're introducing laws at this very moment that will probably give politicians and judges unlimited power to suppress any speech on the internet. They don't call themselves censors, they use euphemisms like "autonomous internet supervision entity".
> I miss "hackers" from the 90s with some actual backbone.
Same. It boggles my mind that I can find people who accept wrongthink as a legitimate concept here on Hacker News. I wonder if those hackers left this place and if so where they are at.
Sorry, but that's not what the word means. By definition, it refers to the interception of communications between others. That's now what's happening here.
"the action of preventing part or the whole of a book, movie, work of art, document, or other kind of communication from being seen or made available to the public, because it is considered to be offensive or harmful, or because it contains information that someone wishes to keep secret, often for political reasons:"
The benefits of censoring misinformation are obvious if you think about it for even a couple minutes.
If you run a community tech support forum and someone starts suggesting deletion of windows32 directory as a solution to other users issues, do you let them do that?
If some neighbor in your bbq party claims to be a doctor and goes on a rant about benefits of microdosing rat poison, do you invite them ever again?
Do you disable spam filter in your mail?
We now have malicious misinformation multiplied to internet scale by modern tech. Add LLMs in the mix and we will drown in it if no measures are taken.
You want to prevent politicians or corporations from abusing or even using censorship at all - good, implement corresponding barriers. Limit scope to clear factual misinfo and not e.g. opinions - yes, sure. Proper transparent process, recourse, rehabilitation, grace of application, oversight - absolutely, implement all of those things.
Make censorship good. Denying it is required makes no sense.
When you first think about it for a bit, it might seem logical, but when you ponder it a bit longer, it doesn't. Censorship essentially comes down to this: some folks believe they know better than others about what those others should want. There's a fitting quote: "A censor is someone who knows more than they think you should." In a world where yesterday's "misinformation" becomes today's fact, you'd have to be pretty clueless to believe that we can determine what is or isn't misinformation.
Also, honestly, the spam filter example (and the rest of your examples) is pretty ridiculous. I don't turn it off, but I'd definitely freak out if my email provider decided to censor my messages for my own good without letting me see them. If nothing else then because of how many false positives I constantly see in my spam folder.
So you are ok with using an automated censor as long as you are able to review (transparency) and revert (recourse) it's results. Like I said, just make it good.
And sure, we can't verify factuality of every claim, but I do believe there is a big enough set where we can. If you accept that all information on internet is fundamentally unverifiable, then of what use is the whole thing?
This is the critical piece of information that I think gets fumbled (and usually on purpose by the media). The word "censor" is being supercharged and abused to apply in cases I don't think it should, and also in absurd cases where no censorship is happening.
"Cases where no censorship is happening": a comedian, to a crowd of thousands, on a Netflix special getting front page treatment, complaining about their jokes being censored for being too edgy.
Cases I don't think it should include instances of websites like Twitter banning hate speech, because the consequence of Twitter banning hate speech is that a user doesn't get to use Twitter anymore. They can still do hate speech in other places, to anyone they want. Whereas the consequences of the ruling communist party banning hate speech is that you well and truly can't do it anywhere, for many reasons.
First reason, every institution will follow the policy of the ruling party the instant it's created. This isn't the case in a country like the usa: despite whining about cancellation, you can get on voat or 4chan and drop the n word all day.
Second reason, there are legislated consequences to violating the censorship of a ruling party ban. In countries like the PRC that probably means some kind of violence such as being jailed, reeducated, or possibly even tortured or shot.
This is nothing like the system we have today. Now for the second part of my thought where I acknowledge some of the subtleties of societal pressure and side effects of what some today call "censorship," or being cancelled, and how despite us not having a ruling communist party, we do seem to have less freedom that one might assume naively above.
With your name attached, drop the n word on Twitter, get banned, no jail, no torture, sure... But you might get doxxed, and you might get banned from other platforms if you're semi famous enough to warrant some place like Facebook needing to purge you for being publicly too toxic to platform. So your actions on Twitter could result in being "censored" on all platforms whose existence depends on surviving in a traditional capitalist mode of production.
But, not just platforms, as we saw entire platforms themselves that try to host this kind of content can be themselves deplatformed by the very internet itself if cloudflare decides not to serve your hate platform.
So it seems in 2023 though nobody has to fear the bullet or the gulag, we've given various corporations so much power that their ability to "censor" is much more capable than ever in history, especially if they collude (which they often seemed forced to do). At least their collusion is much more predictable than the single communist party, whose legislation can drive the will of a single madman depending on their structure. The collusion of "big tech censorship" follows the predictable algorithm of capitalist profit seeking protection. You probably won't randomly get "censored" for criticizing a president's clothes or whatever in this environment.
So in the end though I think people are misusing the word "censor" in a way that diminishes what it means to be censored in the Soviet Union or the PRC, I do think there's a problem with how much power over what's able to be communicated publicly because of monopolization efforts and anti-interop and federalization efforts by large corporations. Don't mistake me for some kind of dude that wishes he could say the n word on Twitter though, my fears are justified by the punishment levied against people for posting pro-union rhetoric, or the doxxing to hostile governments of activists by platforms like Facebook or Twitter, or even just algorithms selectively choosing what people see, usually in favor of culture war rather than showing the reality of the world as a much more mundane place then tik tok would have you believe. This fear is similar to one I've always had about the ability to reach people on television being only available to multi billion dollar corporations.
The actual grim part: It's not an "eventually". Even if the censorship downranks and removes results I disagree with and makes me feel warm and fuzzy and enjoy that my political enemies are gnashing their teeth, it can very well rob me of information it'd be useful to know.
eg. to this day many people live under the impression that Kyle Rittenhouse shot three black people. If asked to estimate how many unarmed blacks get killed by police, Democrat voters give absolutely wild estimates when according to shooting databases the actual number is about 20 a year. According to a poll D voters thought you had a 50% chance of dying if you caught Covid, which is completely nuts even with the earlier, more dangerous strains.
Yet these people think they're well informed and follow the Science™.
If asked to estimate how many unarmed blacks get killed by police, Democrat voters give absolutely wild estimates when according to shooting databases the actual number is about 20 a year.
Uh, no.
By easily findable statistics, the number is more like 225+ a year - shot and killed. And is consistently disproportionate to the number of white victims.
What database are you using? The Washington Post records 12 unarmed black people shot in 2022, 11 in 2021, 18 in 2020, 12 in 2019, 22 in 2018. The 2023 tally thus far is at four. With no year filter, the total number of unarmed black victims is at 157 for the whole 2015-2023 period.
You know, if the conservatives ever come back to power, this precedent that we are creating on justifying censorship will bit us hard, and it will hurt as hell.
It's kind of complicated. The old labels of right and left, conservative vs progressives, and socialist vs capitalists are clearly inadequate nowadays.
Class struggle based left has all but disappeared in the West. And whatever your point of view on American involvement in Ukraine, it is clear that at least the mainstream branches of both parties are largely in accord.
Trump's belligerence on China (justified or not, I am not discussing this) has not only been embraced by the Democrats and NeverТrumpers on the Republican side, but also has been extended far beyond pure economic war, flirting now with proper kinetic war.
Environmental issues, once a solid socialist banner, now have been enthusiastically embraced by Wall Street (I suspect, unfortunately, for not-so-noble reasons beyond banks salivating at the prospect of financing the green transition, and entrepreneurs dreaming about all the opportunities on rebuilding our infrastructure and replacing the entire automobile fleet. Not that is bad in itself, capitalism needs constant growth to be viable, and green rebuilding allows this to happen for a few more decades without utterly destroying our planet).
Civil rights also have been solidly co-opted by the oligarchic class. Both for reasons that the moral behavior and values of the elites are fairly advanced and they want to impose their values as the mainstream values down the throats of the more backward working class and because it is useful as it redirects energy from unionization and other forms of class struggle.
All of our labels and categories simply don't work to understand this complex scenario.
ESG is an easy example of the kind of regulatory moat against competition that classical libertarian analysis routinely complains about.
> Civil rights also have been solidly co-opted by the oligarchic class. Both for reasons that the moral behavior and values of the elites are fairly advanced and they want to impose their values as the mainstream values down the throats of the more backward working class and because it is useful as it redirects energy from unionization and other forms of class struggle.
Worse than that - the elites actively hold luxury beliefs that cause damage at the lower rungs of society but the elites themselves don't quite behave by but which are useful to proclaim as signals of class allegiance and attaining status.
That said, it's been plainly shown that eg. the diversity push is in good part an anti-unionization ploy yet most of the left keeps supporting it.
Their cut of BAT tokens is probably pretty significant. The number of people I met who thought they were "beating" the system by paying Brave 30% of their ad revenue was surprisingly high. I wouldn't be surprised if that surpassed whatever funding DuckDuckGo is able to raise.
It's not two years. Brave Search is based on Tailcat, Cliqz's search engine technology that they kept building after Cliqz proper was shut down. Brave acquired the team and technology 2-3 years ago.
