Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

In 20150101 and 20241201 Dump:Import of release_artist.csv failing because of a release with null artist value #152

Open
ijabz opened this issue Jan 24, 2025 · 6 comments

Comments

@ijabz
Copy link
Collaborator

ijabz commented Jan 24, 2025

Importing data from /home/ubuntu/code/discogs-xml2db/csv-dir/release_artist.csv
Traceback (most recent call last):
  File "postgresql/importcsv.py", line 42, in <module>
    load_csv(os.path.abspath(filename), db)
  File "postgresql/importcsv.py", line 33, in load_csv
    cursor.copy_expert(q, fp)
psycopg2.errors.NotNullViolation: null value in column "artist_id" violates not-null constraint
DETAIL:  Failing row contains (32, 7, null, Koibito & Boku, 1, null, 2, null, Other [Spirits Lifted By], null).
CONTEXT:  COPY release_artist, line 33: "7,,Koibito & Boku,1,,2,,Other [Spirits Lifted By],"
@ijabz ijabz changed the title Download 20150101:Import of release_artist.csv failing becuase of a release with null artist value Download 20150101:Import of release_artist.csv failing because of a release with null artist value Jan 24, 2025
@ijabz ijabz changed the title Download 20150101:Import of release_artist.csv failing because of a release with null artist value In 20150101 Dump:Import of release_artist.csv failing because of a release with null artist value Jan 24, 2025
@ijabz
Copy link
Collaborator Author

ijabz commented Jan 24, 2025

This is the related album https://www.discogs.com/release/7-Moonchildren-Moonchildren-EP?redirected=true

Koibito & Boku are listed as other credit and not clickable, the release artist is MoonChildren.

@ijabz
Copy link
Collaborator Author

ijabz commented Jan 24, 2025

All seem problematic because not release artist for any of these releases.

grep -G "Koibito & Boku" release_artist.csv
7,,Koibito & Boku,1,,2,,Other [Spirits Lifted By],
13,,Koibito & Boku,1,,2,,Other [Spirituality],
5121801,,Koibito & Boku,1,,2,,Other,

Update no that is not isssue they are extra_artists and this is recorded by column 4 =1

@ijabz
Copy link
Collaborator Author

ijabz commented Jan 24, 2025

From original xml

<release id="7" status="Accepted"><artists><artist><id>28</id><name>Moonchildren</name></artist></artists><title>Moonchildren EP</title><labels><label name="Deep4Life" catno="D4L12002" id="9"/></labels><extraartists><artist><id>387624</id><name>Kevin Hodge</name><anv>Ki</anv><role>Lacquer Cut By</role></artist><artist><name>Koibito &amp; Boku</name><role>Other [Spirits Lifted By]</role></artist><artist><id>921599</id><name>Kory And The Gang</name><role>Vocals</role></artist><artist><id>57</id><name>Chris Gray</name><role>Written-By, Producer, Mixed By</role></artist></extraartists><formats><format name="Vinyl" qty="1" text="Clear"><descriptions><description>12"</description><description>33 ⅓ RPM</description><description>EP</description></descriptions></format></formats><genres><genre>Electronic</genre></genres><styles><style>Deep House</style></styles><country>US</country><released>2000-02-00</released><notes>Written, produced and mixed for Mind Massage Music (BMI).

@ijabz
Copy link
Collaborator Author

ijabz commented Jan 24, 2025

grep -E "^[0-9]+,," release_artist.csv |wc -l
731946

So a huge number so I cant work if something broken in this file or not.

@ijabz
Copy link
Collaborator Author

ijabz commented Jan 24, 2025

Compared with 20240601

<release id="7" status="Accepted">
....
<extraartists>
<artist>
<id>387624</id><name>Kevin Hodge</name><anv>Ki</anv><join></join><role>Lacquer Cut By</role><tracks></tracks></artist>
<artist><id>0</id><name>Koibito &amp; Boku</name><anv></anv><join></join><role>Other [Spirits Lifted By]</role><tracks></tracks></artist>
<artist><id>921599</id><name>Kory And The Gang</name><anv></anv><join></join><role>Vocals</role><tracks></tracks></artist><artist><id>57</id><name>Chris Gray</name><anv></anv><join></join>
<role>Written-By, Producer, Mixed By</role><tracks></tracks></artist>
</extraartists>

So the difference is when no have an artist id it used to just set it to zero

`<id>0</id><name>Koibito &amp; Boku</name>`

Now it doesnt set it at all

`<name>Koibito &amp; Boku</name>`

So because the id was nonsense we ignored these rows anyway so personally dont release lose any functionality by removing them from the csv file as being ignored anyway, but woul be better  if did not ignore and could use them somehow.

@ijabz
Copy link
Collaborator Author

ijabz commented Jan 24, 2025

`
 grep -E "^[0-9]+,," -v release_artist.csv > release_artist.tmp; rm release_artist.csv; mv release_artist.tmp release_artist.csv`

@ijabz ijabz changed the title In 20150101 Dump:Import of release_artist.csv failing because of a release with null artist value In 20150101 and 20241201 Dump:Import of release_artist.csv failing because of a release with null artist value Jan 26, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant