Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Output the uploader.py logfiles into the script dir instead of the images dir #18

Open
mattiasostmar opened this issue Jan 18, 2017 · 2 comments

Comments

@mattiasostmar
Copy link

It seems I have lost the external HD when carrying at around during my X-mas holiday. If I don't get it back I cant resolve an important task. Since I store eveything in the script dir on Github it would've been nice to have access to the logs even though the external drive is missing.

Or is there some other simple way to assure this malaise doesn't have to happen again?

@mattiasostmar mattiasostmar changed the title Output the logfiles into the script dir instead of the images dir Output the uploader.py logfiles into the script dir instead of the images dir Jan 18, 2017
@lokal-profil
Copy link
Owner

I think that most of the time storing the logs in the script dir doesn't make sense. If you run multiple uploads with the same script (say you get another 500 images) there would be no easy way of figuring out which log file was related to which image directory and you would have to build in some naming logic to prevent the log file from being overwritten by an unrelated run.

Most of the log files are meant only for temporary usage and should be considered disposable. For the LSH runs I built some final reports though which is something which could be re-usable here. IIRC the reports are:

  • Non-matched files (no or insufficient metadata for these)
  • Non-uploaded files (we made the image description but the upload failed)
  • Successfully uploaded files (id in GLAM system mapped to final filename on Commons, meant for importing the data into their system)

@mattiasostmar
Copy link
Author

Yes, fair points.

This might need further discussion on how to fix - standardizing logging is one of the talking points around code quality/standardizing things tasks on Phabricator

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants