Incorrect or misformatted references

[jalemkul]

Hi Jaime,

Thanks for writing this very useful program. I used it recently and noticed a few issues that I wanted to bring to your attention. There are two main problems that I ran into, but it ended up mangling about 25% of the references in my .bib file, so it required a lot of manual correction.

1. The script seems to directly grab accented ASCII and Greek characters, which then cause typesetting to either fail or leave blanks.

2. In some cases, references are replaced by completely different papers. This was a bit of a headache, because while the screen output warns of poor matches, there were others that were silently replaced with totally different articles (or, in one case, the DOI for a PDB file rather than the actual article associated with the crystal structure itself).

I'm attaching the "old" and "new" versions of my .bib file for your reference. I understand some of this stuff may be beyond the scope of what the program can do, but I wanted to point it out in case there are enhancements that could be made. Thanks again for writing this great tool!

gmx-tutorials-refs.new.txt
gmx-tutorials-refs.old.txt

[jaimergp]

Hello Dr. Lemkul,

thank you for reaching out. I am very glad that you find this tool useful! I am defending my PhD in 10 days so I will not be able to provide any support right now, but I will try to investigate this issue once I am done. I hacked this together for myself in a couple of afternoons so it is far from perfect, but I think that the idea deserves further efforts and I will try to improve it in the next weeks.

For point 1), I haven't experienced such problems in my setup and maybe is a configuration issue, but we could regex the troublesome characters and make sure accents and such are escaped for LaTeX.

For point 2)... Yes, this is one of the things that seemed easy at first. "Just search for the title in CrossRef and retrieve the hit with highest relevance score" should work, but the search API is not perfect and fuzzy queries are not supported AFAIK. So I thought of using the DOI if that's already included in the starting bibliography... but that can be wrong as well... So I tried to apply a string distance metric to warn about possible mismatches in the titles, and thought it would be good enough, but then saw it is not... ¯_(ツ)_/¯

I find this problem very interesting and would like to end up offering a robust solution (I am thinking of some kind of GUI, maybe web-based). I will report back when I have something functional!

Again, thank you for the interest and reporting the problems you have found. I will need regression tests for all these changes, so your reference data is very valuable.

Cheers,
Jaime.

[jalemkul]

Let me know how I can help troubleshoot, and certainly take the time to focus on what is important by defending your dissertation! Good luck, and congratulations in advance.