Readnslate book level checker: sources of the word lists ========================================================= The files in this folder (en.txt, es.txt, de.txt, fr.txt, it.txt, pt.txt, pl.txt, nl.txt, fi.txt, tr.txt and detect.txt) list the 8,000 most common words of each language in their dictionary forms, with the forms that belong to them, and the 200 most common word forms of each language for telling the languages apart. They were made from the sources below and are shared under the Creative Commons Attribution-ShareAlike 4.0 International license (CC BY-SA 4.0): https://creativecommons.org/licenses/by-sa/4.0/ Changes made: word frequencies were added up per dictionary word, forms shared by two dictionary words were divided between them, forms that are mostly names were discounted, and only the 8,000 most common dictionary words were kept. Word frequencies ---------------- wordfreq 3.2.0 by Robyn Speer, https://github.com/rspeer/wordfreq Data files licensed under CC BY-SA 4.0. Files used: large_en, large_es, large_de, large_fr, large_it, large_pt, large_pl, large_nl, large_fi and small_tr. wordfreq contains data from: - Wikipedia, the free encyclopedia (https://www.wikipedia.org) - OPUS OpenSubtitles 2018 (http://opus.nlpl.eu/OpenSubtitles.php), whose data originates from the OpenSubtitles project (http://www.opensubtitles.org/) - SUBTLEX-US, SUBTLEX-UK, SUBTLEX-DE and SUBTLEX-NL, created by Marc Brysbaert et al. and available at http://crr.ugent.be/programs-data/subtitle-frequencies; SUBTLEX is freely available data - Google Books Ngrams (http://books.google.com/ngrams) - The Leeds Internet Corpus, ParaCrawl (https://paracrawl.eu), OSCAR (https://oscar-corpus.com), news, Twitter and Reddit Word forms and dictionary forms ------------------------------- English Wiktionary (https://en.wiktionary.org) by Wiktionary contributors, CC BY-SA 4.0 (and GFDL), in the machine-readable form made by kaikki.org with wiktextract (https://kaikki.org, https://github.com/tatuylonen/wiktextract): extracted on 21 October 2026 from the Wiktionary dump of 1 October 2026. de.txt also ends with one line of German names (people and places) from the same source, which the page uses to tell names from nouns. Forms shared by two dictionary words, and forms that are mostly names --------------------------------------------------------------------- Universal Dependencies treebanks (https://universaldependencies.org): - English: UD English EWT (CC BY-SA 4.0) - Spanish: UD Spanish AnCora (CC BY 4.0), UD Spanish GSD (CC BY-SA 4.0) - German: UD German GSD (CC BY-SA 4.0) - French: UD French GSD, UD French ParisStories (CC BY-SA 4.0) - Italian: UD Italian MarkIT (CC BY 4.0), UD Italian Valico (CC BY-SA 4.0), UD Italian PUD (CC BY-SA 3.0) - Portuguese: UD Portuguese Bosque, UD Portuguese GSD (CC BY-SA 4.0), UD Portuguese Porttinari (CC BY 4.0) - Polish: UD Polish PUD (CC BY-SA 4.0) - Dutch: UD Dutch Alpino (CC BY-SA 4.0) - Finnish: UD Finnish TDT (CC BY-SA 4.0), UD Finnish FTB (CC BY 4.0) - Turkish: UD Turkish BOUN, UD Turkish Kenet, UD Turkish Penn (CC BY-SA 4.0) The page's own code ------------------- PDFs are read with PDF.js 6.4.299 by Mozilla (https://mozilla.github.io/pdf.js/), Apache License 2.0: see ../vendor/pdfjs/LICENSE. The page's font, Libre Baskerville, is under the SIL Open Font License: see ../../assets/fonts/OFL.txt.