Yes you can download all of Wikipedia and yes it can easily fit in a hard drive. Isn't that amazing? Text is incredibly dense compared to images and video. Around 22 GiB for English Wikipedia alone and 56 GiB for the 10 languages I downloaded.

I also have all of Wiktionary in the same hard drive. It's around 16.4 GiB.

you are viewing a single comment's thread
view the rest of the comments

[–] [email protected] 6 points 1 year ago (2 children)

Last time I looked into downloading Wikipedia it said it was 50gb for English text and 100 with images. How'd you get it for half the space?

[–] droning_in_my_ears 9 points 1 year ago (1 children)

It's only the raw text in json line files. No media and no markup. I think I downloaded a compressed dump then used wikiextractor to extract the text.

[–] AbouBenAdhem 2 points 1 year ago

Does it include each article’s edit history, talk page, etc?

[–] ace_garp 2 points 1 year ago* (last edited 1 year ago) (1 children)

The dictionaries for Aard2 are 21gb in .slob compressed format (text only).

[–] [email protected] 2 points 1 year ago (2 children)

No idea what that means. But thank you for adding more info.

[–] ace_garp 3 points 1 year ago

OK yes, some supporting info is: Aard2 is an offline wikipedia app, that uses small compressed data files in .slob format.

[–] [email protected] 1 points 11 months ago

Slob compression is best visualized as putting a sleeping bag into a stuff sack, except it’s all your possessions and you’re stuffing them into an old Chevy Metro