Skip to content

Commit

Permalink
Generate mappings from latest ISO 639-3 table files
Browse files Browse the repository at this point in the history
SIL released a version 2024-10-10 of its ISO 639-3 tables.
  • Loading branch information
LBeaudoux committed Nov 11, 2024
1 parent ef61f20 commit 36be1e8
Show file tree
Hide file tree
Showing 13 changed files with 42 additions and 18 deletions.
9 changes: 6 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -182,7 +182,7 @@ Note that you can use the `is_language` language checker if you don't want to ha

### Checker

The `is_language` function checks if a language value is valid according to ISO 639.
The `is_language` function checks if a language value exists according to ISO 639.

```python
>>> from iso639 import is_language
Expand All @@ -207,14 +207,17 @@ True

## Sources

As of October 23, 2024, `iso639-lang` is based on the latest tables provided by the ISO 639 registration authorities. Please open a new issue if you find that this library uses out-of-date data files.
As of November 11, 2024, `iso639-lang` is based on the latest tables provided by the ISO 639 registration authorities. Please open a new issue if you find that this library uses out-of-date data files.

| Set | Description | Registration Authority | Last Modified |
|--------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------|------------------------|---------------------------------------------------------------------------|
| [Set 1](https://iso639-3.sil.org/sites/iso639-3/files/downloads/iso-639-3.tab) | _two-letter language identifiers for major, mostly national individual languages_ | Infoterm | [2009-09-01](https://www.loc.gov/standards/iso639-2/php/code_changes.php) |
| [Set 2](https://www.loc.gov/standards/iso639-2/ISO-639-2_utf-8.txt) | _three-letter language identifiers for a larger number of widely known individual languages and a number of language groups_ | Library of Congress | [2017-12-21](https://www.loc.gov/standards/iso639-2/php/code_changes.php) |
| [Set 3](https://iso639-3.sil.org/sites/iso639-3/files/downloads/iso-639-3.tab) | _three-letter language identifiers covering all individual languages, including living, extinct and ancient languages_ | SIL International | [2024-04-15](https://iso639-3.sil.org/code_tables/download_tables) |
| [Set 3](https://iso639-3.sil.org/sites/iso639-3/files/downloads/iso-639-3.tab) | _three-letter language identifiers covering all individual languages, including living, extinct and ancient languages_ | SIL International | [2024-10-10](https://iso639-3.sil.org/code_tables/download_tables) |
| [Set 5](http://id.loc.gov/vocabulary/iso639-5.tsv) | _three-letter language identifiers covering a larger set of language groups, living and extinct_ | Library of Congress | [2013-02-11](https://www.loc.gov/standards/iso639-5/changes.php) |

To learn more about how the source tables are processed by the `iso639-lang` library, read the [`generate.py`](https://github.com/LBeaudoux/iso639/blob/master/generate.py) script.

## Contributing

We welcome contributions from the community to help improve this Python library. If you're interested in contributing, please follow the guidelines [here](https://github.com/LBeaudoux/iso639/blob/master/CONTRIBUTING.md).
3 changes: 3 additions & 0 deletions dev_iso639/downloads/iso-639-3-macrolanguages.tab
Original file line number Diff line number Diff line change
Expand Up @@ -447,10 +447,13 @@ zho czh A
zho czo A
zho gan A
zho hak A
zho hnm A
zho hsn A
zho luh A
zho lzh A
zho mnp A
zho nan A
zho sjc A
zho wuu A
zho yue A
zza diq A
Expand Down
11 changes: 6 additions & 5 deletions dev_iso639/downloads/iso-639-3.tab
Original file line number Diff line number Diff line change
Expand Up @@ -906,7 +906,7 @@ bqh I L Baima
bqi I L Bakhtiari
bqj I L Bandial
bqk I L Banda-Mbrès
bql I L Bilakura
bql I L Karian
bqm I L Wumboko
bqn I L Bulgarian Sign Language
bqo I L Balo
Expand Down Expand Up @@ -1489,7 +1489,7 @@ dau I L Dar Sila Daju
dav I L Taita
daw I L Davawenyo
dax I L Dayi
daz I L Dao
daz I L Moi-Wadea
dba I L Bangime
dbb I L Deno
dbd I L Dadiya
Expand Down Expand Up @@ -1530,7 +1530,6 @@ def I L Dezfuli
deg I L Degema
deh I L Dehwari
dei I L Demisa
dek I L Dek
del del del M L Delaware
dem I L Dem
den den den M L Slave (Athapascan)
Expand Down Expand Up @@ -2437,6 +2436,7 @@ hng I L Hungu
hnh I L ǁAni
hni I L Hani
hnj I L Hmong Njua
hnm I L Hainanese
hnn I L Hanunoo
hno I L Northern Hindko
hns I L Caribbean Hindustani
Expand Down Expand Up @@ -3783,6 +3783,7 @@ lud I L Ludian
lue I L Luvale
luf I L Laua
lug lug lug lg I L Ganda
luh I L Leizhou Chinese
lui lui lui I E Luiseno
luj I L Luna
luk I L Lunanakha
Expand Down Expand Up @@ -4128,7 +4129,7 @@ mme I L Mae
mmf I L Mundat
mmg I L North Ambrym
mmh I L Mehináku
mmi I L Musar
mmi I L Hember Avu
mmj I L Majhwar
mmk I L Mukha-Dora
mml I L Man Met
Expand Down Expand Up @@ -4849,7 +4850,6 @@ nsx I L Nsongo
nsy I L Nasal
nsz I L Nisenan
ntd I L Northern Tidung
nte I L Nathembo
ntg I E Ngantangarra
nti I L Natioro
ntj I L Ngaanyatjarra
Expand Down Expand Up @@ -5877,6 +5877,7 @@ siy I L Sivandi
siz I L Siwi
sja I L Epena
sjb I L Sajau Basap
sjc I L Shaojiang Chinese
sjd I L Kildin Sami
sje I L Pite Sami
sjg I L Assangori
Expand Down
9 changes: 7 additions & 2 deletions dev_iso639/downloads/iso-639-3_Name_Index.tab
Original file line number Diff line number Diff line change
Expand Up @@ -937,6 +937,7 @@ bqi Bakhtiari Bakhtiari
bqj Bandial Bandial
bqk Banda-Mbrès Banda-Mbrès
bql Bilakura Bilakura
bql Karian Karian
bqm Wumboko Wumboko
bqn Bulgarian Sign Language Bulgarian Sign Language
bqo Balo Balo
Expand Down Expand Up @@ -1551,6 +1552,7 @@ dav Taita Taita
daw Davawenyo Davawenyo
dax Dayi Dayi
daz Dao Dao
daz Moi-Wadea Moi-Wadea
dba Bangime Bangime
dbb Deno Deno
dbd Dadiya Dadiya
Expand Down Expand Up @@ -1591,7 +1593,6 @@ def Dezfuli Dezfuli
deg Degema Degema
deh Dehwari Dehwari
dei Demisa Demisa
dek Dek Dek
del Delaware Delaware
dem Dem Dem
den Slave (Athapascan) Slave (Athapascan)
Expand Down Expand Up @@ -2562,6 +2563,7 @@ hni Hani Hani
hnj Hmong Njua Hmong Njua
hnj Mong Leng Mong Leng
hnj Mong Njua Mong Njua
hnm Hainanese Hainanese
hnn Hanunoo Hanunoo
hno Northern Hindko Hindko, Northern
hns Caribbean Hindustani Hindustani, Caribbean
Expand Down Expand Up @@ -3982,6 +3984,7 @@ lud Ludian Ludian
lue Luvale Luvale
luf Laua Laua
lug Ganda Ganda
luh Leizhou Chinese Chinese, Leizhou
lui Luiseno Luiseno
luj Luna Luna
luk Lunanakha Lunanakha
Expand Down Expand Up @@ -4339,6 +4342,8 @@ mme Mae Mae
mmf Mundat Mundat
mmg North Ambrym Ambrym, North
mmh Mehináku Mehináku
mmi Amben Amben
mmi Hember Avu Hember Avu
mmi Musar Musar
mmj Majhwar Majhwar
mmk Mukha-Dora Mukha-Dora
Expand Down Expand Up @@ -5086,7 +5091,6 @@ nsx Nsongo Nsongo
nsy Nasal Nasal
nsz Nisenan Nisenan
ntd Northern Tidung Tidung, Northern
nte Nathembo Nathembo
ntg Ngantangarra Ngantangarra
nti Natioro Natioro
ntj Ngaanyatjarra Ngaanyatjarra
Expand Down Expand Up @@ -6161,6 +6165,7 @@ siy Sivandi Sivandi
siz Siwi Siwi
sja Epena Epena
sjb Sajau Basap Sajau Basap
sjc Shaojiang Chinese Chinese, Shaojiang
sjd Kildin Sami Sami, Kildin
sje Pite Sami Sami, Pite
sjg Assangori Assangori
Expand Down
12 changes: 12 additions & 0 deletions dev_iso639/downloads/iso-639-3_Retirements.tab
Original file line number Diff line number Diff line change
Expand Up @@ -372,6 +372,18 @@ uun Kulon-Pazeh S Split into Kulon [uon] and Pazeh [pzh] 2022-01-20
wrd Warduji N 2022-01-20
wya Wyandot S Split into Wyandot [wyn] and Wendat [wdt] 2022-01-20
ajp South Levantine Arabic M apc 2023-01-20
kgm Karipúna D plu 2023-01-20
nom Nocamán M cbr 2023-01-20
pmk Pamlico M crr 2023-01-20
prp Parsi M guj 2023-01-20
slq Salchuq N 2023-01-20
szd Seru M umi 2023-01-20
tmk Northwestern Tamang M tdg 2023-01-20
tpw Tupí D tpn 2023-01-20
xss Assan M zko 2023-01-20
zkb Koibal D kjh 2023-01-20
zua Zeem S Split into [tvi] Tulai, [dyr] Dyarim, [dsk] Dokshi, [cxh] Cha'ari, and [zem] Zeem 2023-01-20
ksa Shuwa-Zamani S Split into [rsw] Rishiwa and [izm] Kizamani 2023-01-23
plj Polci S Split into Pesse [pze], Nyamzak-Dir-Mbarimi [nzr], Zul [zlu], and Buli [uly] 2023-01-23
dek Dek D sqm 2024-09-09
nte Nathembo M eko 2024-09-09
2 changes: 1 addition & 1 deletion iso639/data/iso-639.json

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion iso639/data/iso-639_deprecated.json

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion iso639/data/iso-639_langs.json

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion iso639/data/iso-639_macro.json

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion iso639/data/iso-639_other_names.json

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion iso639/data/iso-639_ref_name.json

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion iso639/data/iso-639_scope.json

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion iso639/data/iso-639_type.json

Large diffs are not rendered by default.

0 comments on commit 36be1e8

Please sign in to comment.