With only 2 weeks left till May 4th (be with you), the internet is bursting with excitement over all the work that needs to be done before Yahoo Answers finally 404s.
From scheduling a 2nd COVID vaccine to your annual panic attack at missing the tax filing deadline (you probably didn’t, it was extended to May 17 in the U.S.), there is nothing short of a lengthy agenda for everyone ahead of the shutdown of this iconic website.
Here at Diffbot, we obsess over sites full of such essential facts as the answers to “Is Christian Bale a Christian since his name is Christian?” and “How is babby formed?”. So it should be of no surprise to hear that we scraped it all (at least as many as we could find on their English site).
12,118 questions, along with all the meticulously crafted answers to each shared only by the internet’s best, available as a JSON download for only $1*.
Why $1? Because this is a bona fide piece of internet history. Because I’m too lazy to figure out how to set this up as an NFT. Because it’s all I can honestly ask for how ridiculously easy it was for me to crawl the site.
Seriously, it took all of entering 4 fields in Diffbot Crawl, setting it to automatically extract using the Discussion API, and hitting “Submit”.
Not sure what you’re getting? Here’s a sample discussion:
{
"numPages": 1,
"humanLanguage": "en",
"confidence": 0.27939321284648105,
"diffbotUri": "discussion|3|1662932736",
"pageUrl": "https://answers.yahoo.com/question/index?qid=20210331193044AADVb1B",
"numPosts": 11,
"type": "discussion",
"title": "is eminem overrated ?",
"posts": [
{
"date": "Tue, 30 Mar 2021 00:00:00 GMT",
"humanLanguage": "en",
"author": "Anonymous",
"diffbotUri": "post|3|824248295",
"html": "",
"pageUrl": "https://answers.yahoo.com/question/index?qid=20210331193044AADVb1B",
"id": 0,
"text": "is eminem overrated?",
"type": "post"
},
{
"date": "Tue, 06 Apr 2021 00:00:00 GMT",
"humanLanguage": "en",
"author": "ronald",
"diffbotUri": "post|3|1589160025",
"html": "Highly, never heard anyone banging eminem in my hood EVER! K-Rino would eat him alive",
"pageUrl": "https://answers.yahoo.com/question/index?qid=20210331193044AADVb1B",
"id": 1,
"text": "Highly, never heard anyone banging eminem in my hood EVER! K-Rino would eat him alive",
"type": "post",
"parentId": 0
},
{
"date": "Tue, 06 Apr 2021 00:00:00 GMT",
"humanLanguage": "en",
"author": "Eric",
"authorUrl": "https://answers.yahoo.com/activity/questions?show=C9JfxoZ8aa",
"diffbotUri": "post|3|-1595207957",
"html": "Not even remotely, the man is incredible and arguably the greatest rapper that has ever lived
",
"pageUrl": "https://answers.yahoo.com/question/index?qid=20210331193044AADVb1B",
"id": 2,
"text": "Not even remotely, the man is incredible and arguably the greatest rapper that has ever lived",
"type": "post"
},
{
"date": "Tue, 06 Apr 2021 00:00:00 GMT",
"humanLanguage": "en",
"author": "JohnnyC",
"authorUrl": "https://answers.yahoo.com/activity/questions?show=CZBR5ZY6aa",
"diffbotUri": "post|3|-349370742",
"html": "Not at all... He’s the goat 🐐
",
"pageUrl": "https://answers.yahoo.com/question/index?qid=20210331193044AADVb1B",
"id": 3,
"text": "Not at all... He’s the goat 🐐",
"type": "post"
},
{
"date": "Tue, 06 Apr 2021 00:00:00 GMT",
"humanLanguage": "en",
"author": "Anonymous",
"diffbotUri": "post|3|178511364",
"html": "lol, no.
\ndon't be so fecking stewwwpid ^_^
\nthe LAST fool who said/thought that, was demolished by Slim Shady.
\neach time someone steps up to him he's 'dealt with' accordingly.
\nhe will - feck - your - s*** - up ! ^_^
\nYOUR s*** mutha phecka ! LOL
\nj/k bro
\nfinal answer - No! lol
\n...Show more
",
"pageUrl": "https://answers.yahoo.com/question/index?qid=20210331193044AADVb1B",
"id": 4,
"text": "lol, no.\ndon't be so fecking stewwwpid ^_^\nthe LAST fool who said/thought that, was demolished by Slim Shady.\neach time someone steps up to him he's 'dealt with' accordingly.\nhe will - feck - your - s*** - up ! ^_^\nYOUR s*** mutha phecka ! LOL\nj/k bro\nfinal answer - No! lol\n...Show more",
"type": "post",
"parentId": 3
},
{
"date": "Tue, 06 Apr 2021 00:00:00 GMT",
"humanLanguage": "en",
"author": "Anonymous",
"diffbotUri": "post|3|1797179185",
"html": "all popular mainstream rappers are overated.
",
"pageUrl": "https://answers.yahoo.com/question/index?qid=20210331193044AADVb1B",
"id": 5,
"text": "all popular mainstream rappers are overated.",
"type": "post",
"parentId": 3
},
{
"date": "Tue, 30 Mar 2021 00:00:00 GMT",
"humanLanguage": "en",
"author": "?",
"diffbotUri": "post|3|311380370",
"html": "Yea he wack..only good song is deat stan..
",
"pageUrl": "https://answers.yahoo.com/question/index?qid=20210331193044AADVb1B",
"id": 6,
"text": "Yea he wack..only good song is deat stan..",
"type": "post",
"parentId": 3
},
{
"date": "Tue, 30 Mar 2021 00:00:00 GMT",
"humanLanguage": "en",
"author": "Anonymous",
"diffbotUri": "post|3|-2018805831",
"html": "No, it's currently underrated,nowadays everybody hate him,but the God will wipe their ***.
",
"pageUrl": "https://answers.yahoo.com/question/index?qid=20210331193044AADVb1B",
"id": 7,
"text": "No, it's currently underrated,nowadays everybody hate him,but the God will wipe their ***.",
"type": "post",
"parentId": 3
},
{
"date": "Tue, 30 Mar 2021 00:00:00 GMT",
"humanLanguage": "en",
"author": "Big Sosa “The Jefe”",
"diffbotUri": "post|3|-1164416800",
"html": "He’s the W O A T Worst Of All Time I’ve hated him since I discovered him online when I was 8.
\nFun Fact: My least Favorite song in music history as a whole is Just Lose It cuz of the beat, Michael Jackson sucks and Eminem is talking about him and Michael Jackson is worse than Em
",
"pageUrl": "https://answers.yahoo.com/question/index?qid=20210331193044AADVb1B",
"id": 8,
"text": "He’s the W O A T Worst Of All Time I’ve hated him since I discovered him online when I was 8.\n\nFun Fact: My least Favorite song in music history as a whole is Just Lose It cuz of the beat, Michael Jackson sucks and Eminem is talking about him and Michael Jackson is worse than Em",
"type": "post",
"parentId": 3
},
{
"date": "Tue, 30 Mar 2021 00:00:00 GMT",
"humanLanguage": "en",
"author": "blank",
"authorUrl": "https://answers.yahoo.com/activity/questions?show=FvYqKZSMaa",
"diffbotUri": "post|3|1536839535",
"html": "I feel no Rapper is overrated. If people like the Rapper then they like that Rapper. So I am going to say no.
",
"pageUrl": "https://answers.yahoo.com/question/index?qid=20210331193044AADVb1B",
"id": 9,
"text": "I feel no Rapper is overrated. If people like the Rapper then they like that Rapper. So I am going to say no.",
"type": "post"
},
{
"date": "Tue, 30 Mar 2021 00:00:00 GMT",
"humanLanguage": "en",
"author": "ArsosA",
"authorUrl": "https://answers.yahoo.com/activity/questions?show=6LzwdG7Paa",
"diffbotUri": "post|3|356562743",
"html": "I have my monologues in my head with him, I don't know how I feel about him. I went through my Eminem phase. All he put out after MMLP2 didn't live up to the expectations. I'm 50/50 on his discography. He has some dope albums like The Slim Shady LP, The Marshall Mathers LP, The Eminem Show, Recovery, MMLP2 and then some less good albums like Infinite (which I thought it was still good), Encore, Relapse, Revival, Kamikaze, and Music To Be Murdered By sides A and B. All in all, I'm a stan but I can't stand him sometimes. Still a good rapper but overrated.
\n...Show more
",
"pageUrl": "https://answers.yahoo.com/question/index?qid=20210331193044AADVb1B",
"id": 10,
"text": "I have my monologues in my head with him, I don't know how I feel about him. I went through my Eminem phase. All he put out after MMLP2 didn't live up to the expectations. I'm 50/50 on his discography. He has some dope albums like The Slim Shady LP, The Marshall Mathers LP, The Eminem Show, Recovery, MMLP2 and then some less good albums like Infinite (which I thought it was still good), Encore, Relapse, Revival, Kamikaze, and Music To Be Murdered By sides A and B. All in all, I'm a stan but I can't stand him sometimes. Still a good rapper but overrated.\n...Show more",
"type": "post"
}
],
"participants": 8,
}
The full dataset is so incorrigibly massive that I can barely get past 10 frames per second viewing it on my decidedly not-M1 Macbook.
Diffbot Crawl also runs discussion text through our own natural language processing, extrapolating tags in each discussion for easier analysis.
Here’re the top 10 tags** (all 9286 unedited tags are available to download):
- United States of America – 1517
- vaccine – 540
- Seraphim of Sarov – 448
- People – 445
- United Kingdom – 413
- People’s Republic of China – 405
- Canada – 351
- child – 325
- tax – 323
- Donald Trump – 307
My only comment on these tags — Seraphim of Sarov, a renowned Russian saint, is quoted along with more instructions to “escape the unforgivable mark of the beast” as answer spam throughout hundreds of these posts. Someone had a lot of time in their hands.
You might be asking, “what can I possibly do with 91.7 megabytes of Grade A certified unadulterated internet commentary?”
A few (66% serious) ideas:
- Conspiracy theory research
- Endless gold mine of meme inspiration
- Support the Equal Justice Initiative
Oh, did we not mention? All proceeds will be donated to the Equal Justice Initiative, a non-profit organization committed to ending mass incarceration and excessive punishment in the United States, to challenging racial and economic injustice, and to protecting basic human rights for the most vulnerable people in American society.
Get the full download here, or learn more about Diffbot Crawl and Extract.
* 1 Not actually for sale. All proceeds will be donated to the Equal Justice Initiative.
** 2 With some discretionary filtering of tags like “Yahoo! Answers”, “Yahoo”, and “Eastern Time Zone”. The lattermost being due to the appearance of the phrase “Eastern Time” in the sitewide shut down announcement banner.
Diffbot is not affiliated or associated in any way with Yahoo! or its parent company, Verizon Media.
You must be logged in to post a comment.