Add new http_get extra function with header & params

[ahuarte47]

This PR implements #7

[lmangani]

Thanks @ahuarte47! I'm trying to understand why the MINGW buileder fails as we seem to correctly link everything in CMake....

[lmangani]

since the http_post function seems fine I guess the issue is in the differences between the two not using linked httplib functions somehow. I can take a look later tonight.

[lmangani]

I couldn't find a way to make MGW happy from the linker without code changes. Since the only difference appear to be the params, a method to avoid this could be to build the URL with parameters manually before making the request, but I did not try.

An easier option - How about excluding windows_mgw from builds @ahuarte47 ?

[ahuarte47]

I do not see any problem building the URL with parameters manually. I can change code this weekend, or if you prefer you can modify yourself.

[ahuarte47]

Hi @lmangani I do not understand why MGW can not build the extension now. Please, can you share the error message?
httplib is expanding the path with parameters as well, so, your idea is great, but the logic in httplib seems more elaborate.

Maybe is the new template function to convert input parameters the guilty?

[lmangani]

Hola Muy Buenas @ahuarte47 I also do not understand where all the unresolved symbols come from. I wonder if it has to to with the latest version of DuckDB since our code didn't really change much.

libcrypto.a(libcrypto-lib-dso_win32.obj) : error LNK2001: unresolved external symbol sprintf [D:\a\duckdb-extension-httpclient\duckdb-extension-httpclient\build\release\extension\parquet\parquet_loadable_extension.vcxproj]
libcrypto.a(libcrypto-lib-rsaz_exp.obj) : error LNK2001: unresolved external symbol ___chkstk_ms [D:\a\duckdb-extension-httpclient\duckdb-extension-httpclient\build\release\extension\parquet\parquet_loadable_extension.vcxproj]
libcrypto.a(libcrypto-lib-e_capi.obj) : error LNK2001: unresolved external symbol ___chkstk_ms [D:\a\duckdb-extension-httpclient\duckdb-extension-httpclient\build\release\extension\parquet\parquet_loadable_extension.vcxproj]
libcrypto.a(libcrypto-lib-cmp_util.obj) : error LNK2001: unresolved external symbol ___chkstk_ms [D:\a\duckdb-extension-httpclient\duckdb-extension-httpclient\build\release\extension\parquet\parquet_loadable_extension.vcxproj]
libcrypto.a(libcrypto-lib-bn_exp.obj) : error LNK2001: unresolved external symbol ___chkstk_ms [D:\a\duckdb-extension-httpclient\duckdb-extension-httpclient\build\release\extension\parquet\parquet_loadable_extension.vcxproj]
libcrypto.a(libcrypto-lib-curve448.obj) : error LNK2001: unresolved external symbol ___chkstk_ms [D:\a\duckdb-extension-httpclient\duckdb-extension-httpclient\build\release\extension\parquet\parquet_loadable_extension.vcxproj]
libcrypto.a(libcrypto-lib-cryptlib.obj) : error LNK2001: unresolved external symbol ___chkstk_ms [D:\a\duckdb-extension-httpclient\duckdb-extension-httpclient\build\release\extension\parquet\parquet_loadable_extension.vcxproj]
libcrypto.a(libcrypto-lib-getenv.obj) : error LNK2001: unresolved external symbol ___chkstk_ms [D:\a\duckdb-extension-httpclient\duckdb-extension-httpclient\build\release\extension\parquet\parquet_loadable_extension.vcxproj]
libcrypto.a(libcrypto-lib-err_prn.obj) : error LNK2001: unresolved external symbol ___chkstk_ms [D:\a\duckdb-extension-httpclient\duckdb-extension-httpclient\build\release\extension\parquet\parquet_loadable_extension.vcxproj]
libcrypto.a(libcrypto-lib-o_fopen.obj) : error LNK2001: unresolved external symbol ___chkstk_ms [D:\a\duckdb-extension-httpclient\duckdb-extension-httpclient\build\release\extension\parquet\parquet_loadable_extension.vcxproj]
libssl.a(libssl-lib-tls_common.obj) : error LNK2001: unresolved external symbol ___chkstk_ms [D:\a\duckdb-extension-httpclient\duckdb-extension-httpclient\build\release\extension\parquet\parquet_loadable_extension.vcxproj]
libssl.a(libssl-lib-extensions_srvr.obj) : error LNK2001: unresolved external symbol ___chkstk_ms [D:\a\duckdb-extension-httpclient\duckdb-extension-httpclient\build\release\extension\parquet\parquet_loadable_extension.vcxproj]
libcrypto.a(libcrypto-lib-o_dir.obj) : error LNK2001: unresolved external symbol ___chkstk_ms [D:\a\duckdb-extension-httpclient\duckdb-extension-httpclient\build\release\extension\parquet\parquet_loadable_extension.vcxproj]
libcrypto.a(libcrypto-lib-srp_vfy.obj) : error LNK2001: unresolved external symbol ___chkstk_ms [D:\a\duckdb-extension-httpclient\duckdb-extension-httpclient\build\release\extension\parquet\parquet_loadable_extension.vcxproj]
libcrypto.a(libcrypto-lib-v3_utl.obj) : error LNK2019: unresolved external symbol sscanf referenced in function ipv4_from_asc [D:\a\duckdb-extension-httpclient\duckdb-extension-httpclient\build\release\extension\parquet\parquet_loadable_extension.vcxproj]
libcrypto.a(libcrypto-lib-http_lib.obj) : error LNK2001: unresolved external symbol sscanf [D:\a\duckdb-extension-httpclient\duckdb-extension-httpclient\build\release\extension\parquet\parquet_loadable_extension.vcxproj]
libcrypto.a(libcrypto-lib-bio_addr.obj) : error LNK2019: unresolved external symbol WspiapiGetNameInfo referenced in function addr_strings.part.1 [D:\a\duckdb-extension-httpclient\duckdb-extension-httpclient\build\release\extension\parquet\parquet_loadable_extension.vcxproj]
libcrypto.a(libcrypto-lib-bio_addr.obj) : error LNK2019: unresolved external symbol gai_strerrorA referenced in function addr_strings.part.1 [D:\a\duckdb-extension-httpclient\duckdb-extension-httpclient\build\release\extension\parquet\parquet_loadable_extension.vcxproj]
libcrypto.a(libcrypto-lib-bio_addr.obj) : error LNK2019: unresolved external symbol WspiapiFreeAddrInfo referenced in function BIO_ADDRINFO_free [D:\a\duckdb-extension-httpclient\duckdb-extension-httpclient\build\release\extension\parquet\parquet_loadable_extension.vcxproj]
libcrypto.a(libcrypto-lib-bio_addr.obj) : error LNK2019: unresolved external symbol WspiapiGetAddrInfo referenced in function BIO_lookup [D:\a\duckdb-extension-httpclient\duckdb-extension-httpclient\build\release\extension\parquet\parquet_loadable_extension.vcxproj]
libcrypto.a(libcrypto-lib-cryptlib.obj) : error LNK2019: unresolved external symbol __imp__vsnwprintf referenced in function OPENSSL_showfatal [D:\a\duckdb-extension-httpclient\duckdb-extension-httpclient\build\release\extension\parquet\parquet_loadable_extension.vcxproj]
libcrypto.a(libcrypto-lib-cryptlib.obj) : error LNK2019: unresolved external symbol __imp__vsnprintf referenced in function OPENSSL_showfatal [D:\a\duckdb-extension-httpclient\duckdb-extension-httpclient\build\release\extension\parquet\parquet_loadable_extension.vcxproj]
libcrypto.a(libcrypto-lib-a_time.obj) : error LNK2019: unresolved external symbol __imp__timezone referenced in function ossl_asn1_string_to_time_t [D:\a\duckdb-extension-httpclient\duckdb-extension-httpclient\build\release\extension\parquet\parquet_loadable_extension.vcxproj]
D:\a\duckdb-extension-httpclient\duckdb-extension-httpclient\build\release\extension\parquet\parquet.duckdb_extension : fatal error LNK1120: 10 unresolved externals [D:\a\duckdb-extension-httpclient\duckdb-extension-httpclient\build\release\extension\parquet\parquet_loadable_extension.vcxproj]

[ahuarte47]

Hola Muy Buenas @ahuarte47 I also do not understand where all the unresolved symbols come from. I wonder if it has to to with the latest version of DuckDB since our code didn't really change much.

libcrypto.a(libcrypto-lib-dso_win32.obj) : error LNK2001: unresolved external symbol sprintf [D:\a\duckdb-extension-httpclient\duckdb-extension-httpclient\build\release\extension\parquet\parquet_loadable_extension.vcxproj]
libcrypto.a(libcrypto-lib-rsaz_exp.obj) : error LNK2001: unresolved external symbol ___chkstk_ms [D:\a\duckdb-extension-httpclient\duckdb-extension-httpclient\build\release\extension\parquet\parquet_loadable_extension.vcxproj]
libcrypto.a(libcrypto-lib-e_capi.obj) : error LNK2001: unresolved external symbol ___chkstk_ms [D:\a\duckdb-extension-httpclient\duckdb-extension-httpclient\build\release\extension\parquet\parquet_loadable_extension.vcxproj]
libcrypto.a(libcrypto-lib-cmp_util.obj) : error LNK2001: unresolved external symbol ___chkstk_ms [D:\a\duckdb-extension-httpclient\duckdb-extension-httpclient\build\release\extension\parquet\parquet_loadable_extension.vcxproj]
libcrypto.a(libcrypto-lib-bn_exp.obj) : error LNK2001: unresolved external symbol ___chkstk_ms [D:\a\duckdb-extension-httpclient\duckdb-extension-httpclient\build\release\extension\parquet\parquet_loadable_extension.vcxproj]
libcrypto.a(libcrypto-lib-curve448.obj) : error LNK2001: unresolved external symbol ___chkstk_ms [D:\a\duckdb-extension-httpclient\duckdb-extension-httpclient\build\release\extension\parquet\parquet_loadable_extension.vcxproj]
libcrypto.a(libcrypto-lib-cryptlib.obj) : error LNK2001: unresolved external symbol ___chkstk_ms [D:\a\duckdb-extension-httpclient\duckdb-extension-httpclient\build\release\extension\parquet\parquet_loadable_extension.vcxproj]
libcrypto.a(libcrypto-lib-getenv.obj) : error LNK2001: unresolved external symbol ___chkstk_ms [D:\a\duckdb-extension-httpclient\duckdb-extension-httpclient\build\release\extension\parquet\parquet_loadable_extension.vcxproj]
libcrypto.a(libcrypto-lib-err_prn.obj) : error LNK2001: unresolved external symbol ___chkstk_ms [D:\a\duckdb-extension-httpclient\duckdb-extension-httpclient\build\release\extension\parquet\parquet_loadable_extension.vcxproj]
libcrypto.a(libcrypto-lib-o_fopen.obj) : error LNK2001: unresolved external symbol ___chkstk_ms [D:\a\duckdb-extension-httpclient\duckdb-extension-httpclient\build\release\extension\parquet\parquet_loadable_extension.vcxproj]
libssl.a(libssl-lib-tls_common.obj) : error LNK2001: unresolved external symbol ___chkstk_ms [D:\a\duckdb-extension-httpclient\duckdb-extension-httpclient\build\release\extension\parquet\parquet_loadable_extension.vcxproj]
libssl.a(libssl-lib-extensions_srvr.obj) : error LNK2001: unresolved external symbol ___chkstk_ms [D:\a\duckdb-extension-httpclient\duckdb-extension-httpclient\build\release\extension\parquet\parquet_loadable_extension.vcxproj]
libcrypto.a(libcrypto-lib-o_dir.obj) : error LNK2001: unresolved external symbol ___chkstk_ms [D:\a\duckdb-extension-httpclient\duckdb-extension-httpclient\build\release\extension\parquet\parquet_loadable_extension.vcxproj]
libcrypto.a(libcrypto-lib-srp_vfy.obj) : error LNK2001: unresolved external symbol ___chkstk_ms [D:\a\duckdb-extension-httpclient\duckdb-extension-httpclient\build\release\extension\parquet\parquet_loadable_extension.vcxproj]
libcrypto.a(libcrypto-lib-v3_utl.obj) : error LNK2019: unresolved external symbol sscanf referenced in function ipv4_from_asc [D:\a\duckdb-extension-httpclient\duckdb-extension-httpclient\build\release\extension\parquet\parquet_loadable_extension.vcxproj]
libcrypto.a(libcrypto-lib-http_lib.obj) : error LNK2001: unresolved external symbol sscanf [D:\a\duckdb-extension-httpclient\duckdb-extension-httpclient\build\release\extension\parquet\parquet_loadable_extension.vcxproj]
libcrypto.a(libcrypto-lib-bio_addr.obj) : error LNK2019: unresolved external symbol WspiapiGetNameInfo referenced in function addr_strings.part.1 [D:\a\duckdb-extension-httpclient\duckdb-extension-httpclient\build\release\extension\parquet\parquet_loadable_extension.vcxproj]
libcrypto.a(libcrypto-lib-bio_addr.obj) : error LNK2019: unresolved external symbol gai_strerrorA referenced in function addr_strings.part.1 [D:\a\duckdb-extension-httpclient\duckdb-extension-httpclient\build\release\extension\parquet\parquet_loadable_extension.vcxproj]
libcrypto.a(libcrypto-lib-bio_addr.obj) : error LNK2019: unresolved external symbol WspiapiFreeAddrInfo referenced in function BIO_ADDRINFO_free [D:\a\duckdb-extension-httpclient\duckdb-extension-httpclient\build\release\extension\parquet\parquet_loadable_extension.vcxproj]
libcrypto.a(libcrypto-lib-bio_addr.obj) : error LNK2019: unresolved external symbol WspiapiGetAddrInfo referenced in function BIO_lookup [D:\a\duckdb-extension-httpclient\duckdb-extension-httpclient\build\release\extension\parquet\parquet_loadable_extension.vcxproj]
libcrypto.a(libcrypto-lib-cryptlib.obj) : error LNK2019: unresolved external symbol __imp__vsnwprintf referenced in function OPENSSL_showfatal [D:\a\duckdb-extension-httpclient\duckdb-extension-httpclient\build\release\extension\parquet\parquet_loadable_extension.vcxproj]
libcrypto.a(libcrypto-lib-cryptlib.obj) : error LNK2019: unresolved external symbol __imp__vsnprintf referenced in function OPENSSL_showfatal [D:\a\duckdb-extension-httpclient\duckdb-extension-httpclient\build\release\extension\parquet\parquet_loadable_extension.vcxproj]
libcrypto.a(libcrypto-lib-a_time.obj) : error LNK2019: unresolved external symbol __imp__timezone referenced in function ossl_asn1_string_to_time_t [D:\a\duckdb-extension-httpclient\duckdb-extension-httpclient\build\release\extension\parquet\parquet_loadable_extension.vcxproj]
D:\a\duckdb-extension-httpclient\duckdb-extension-httpclient\build\release\extension\parquet\parquet.duckdb_extension : fatal error LNK1120: 10 unresolved externals [D:\a\duckdb-extension-httpclient\duckdb-extension-httpclient\build\release\extension\parquet\parquet_loadable_extension.vcxproj]

Hi Lorenzo, can you run make clean + make test to refresh the complete build?

[lmangani]

Hey @ahuarte47

Hi Lorenzo, can you run make clean + make test to refresh the complete build?

The errors are from the Github action builder, not from my local setup. (I don't have a Windows MINIGW env to test with)

[lmangani]

@ahuarte47 i excluded the windows_amd64_mingw for the time being. We can figure this out later but since 99.9% of the userbase for this extension is on Linux and OSX releasing your new features seems more important. If you agree just pull main back into the branch and it should pass. I'll take care of doing the same on the extensions repository. Vale?

[ahuarte47]

Ah, thanks, sorry for my misunderstanding. Ok I see your commit 640fc62

[ahuarte47]

@ahuarte47 i excluded the windows_amd64_mingw for the time being. We can figure this out later but since 99.9% of the userbase for this extension is on Linux and OSX releasing your new features seems more important. If you agree just pull main back into the branch and it should pass. I'll take care of doing the same on the extensions repository. Vale?

Yeah!!! I totaly agree! Thanks @lmangani