Transaction XTB error

[rubengr]

Hi

I'm testing you code, but get this error:

npm run start xtb

> export-to-ghostfolio@0.18.2 start
> tsx ./src/manual.ts xtb

[i] Starting Export to Ghostfolio v0.18.2
[i] Restored 0 ISIN-symbol pairs and 0 symbols from cache..
[i] Processing file using XTB converter
[i] Read CSV file. Start processing..
{
  iD: '629854440',
  type: 'interest',
  time: '04.10.2024 11:03:40',
  symbol: '',
  comment: 'Free-funds Interest Tax 2024-09',
  amount: -0.03
}
{
  iD: '629854424',
  type: 'interest',
  time: '04.10.2024 11:03:39',
  symbol: '',
  comment: 'Free-funds Interest 2024-09',
  amount: 0.1
}
{
  iD: '624529037',
  type: 'buy',
  time: '27.09.2024 12:22:31',
  symbol: 'TXT.PL',
  comment: 'OPEN BUY 5 @ 74.10',
  amount: -87.09
}
[d]     getSecurity(): Found 0 matches by symbol TXT.PL
[d]     getSecurity(): No initial match found, trying by symbol TXT.PL
[d]     getSecurity(): No match found for TXT.PL, trying to symbol TXT
[i] ░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ 0% | ETA 2s | Duration: 1s | 2/765We expected a redirect to guce.yahoo.com/We'll try to continue anyway - you can safely ignore this if the request succeeds
[d]             getSymbolsByQuery(): Quote 'TXT' has no symbol at Yahoo Finance undefined. Skipping..
[d]             getSymbolsByQuery(): Quote 'TXT' has no symbol at Yahoo Finance undefined. Skipping..
[d]     getSecurity(): Match found for TXT.PL
[i] ░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ 0% | ETA 2s | Duration: 2s | 2/765{
  iD: '623424924',
  type: 'Withholding tax',
  time: '26.09.2024 11:00:23',
  symbol: 'FB.US',
  comment: 'FB.US USD WHT 15%',
  amount: -0.33
}
file:///C:/Users/Ruben/Documents/Ruben/Ghostfolio/Export-To-Ghostfolio/src/converters/xtbConverter.ts:1
var __defProp=Object.defineProperty;var __name=(target,value)=>__defProp(target,"name",{value,configurable:true});impor.

TypeError: Cannot read properties of null (reading '1')
    at <anonymous> (c:\Users\Ruben\Documents\Ruben\Ghostfolio\Export-To-Ghostfolio\src\converters\xtbConverter.ts:114:4)
Node.js v20.11.0

[dickwolff]

Thanks for reporting this. I will look into it when I have time!

[dickwolff]

@rubengr I've just looked at the issue. The Withholding tax record is new for the tool, so it goes broke for some technical reason.

I assume this record is not standalone, but rather connected to another BUY/SELL record. Could you maybe send me both lines from your export, so I can use that to create a working solution?

[dickwolff]

@rubengr Could you check my latest message? Thanks!

[rubengr]

@rubengr Could you check my latest message? Thanks!

sorry for late reply. I think is related to taxes of dividends. For example:
`

<style> </style>

ID	Type	Time	Symbol	Comment	Amount
623424924	Withholding tax	26.09.2024 11:00:23	FB.US	FB.US USD WHT 15%	-0.33
623424923	Dividend	26.09.2024 11:00:23	FB.US	FB.US USD 0.5000/ SHR	2.23

`

[dickwolff]

Sorry for the late reply. I was busy finishing things up last year. Now I have more time for this project and I will look into this issue today!

[dickwolff]

@rubengr After looking into it, it seems the XTB converter did not have support for dividends at all. I've added this just now, and used your input as test data.

Can you test if this works for you? You can clone this branch and run it yourself, or use this Docker image to verify. Please let me know if all works as expected so I can merge this.

[rubengr]

Get this error:

PS C:\Users\Ruben\Downloads\Export-To-Ghostfolio> npm run start xtb

> export-to-ghostfolio@0.20.1 start
> tsx ./src/manual.ts xtb

[i] Starting Export to Ghostfolio v0.20.1
[i] Restored 7 ISIN-symbol pairs and 7 symbols from cache..
[i] Processing file using XTB converter
[i] Read CSV file. Start processing..
file:///C:/Users/Ruben/Downloads/Export-To-Ghostfolio/src/converters/xtbConverter.ts:2
`);bar1.increment();continue}let feeAmount=0;if(record.type.toLocaleLowerCase()==="dividend"){const taxRecord=this.lookupDividendTaxRecord(records,idx);if(;                                                                                                                                                           ^
TypeError: Cannot read properties of undefined (reading 'toLocaleLowerCase')
    at <anonymous> (c:\Users\Ruben\Downloads\Export-To-Ghostfolio\src\converters\xtbConverter.ts:188:57)
    at Array.some (<anonymous>)
    at XtbConverter.isIgnoredRecord (c:\Users\Ruben\Downloads\Export-To-Ghostfolio\src\converters\xtbConverter.ts:188:35)
    at <anonymous> (c:\Users\Ruben\Downloads\Export-To-Ghostfolio\src\converters\xtbConverter.ts:88:26)
    at Parser.<anonymous> (file:///C:/Users/Ruben/Downloads/Export-To-Ghostfolio/node_modules/csv-parse/lib/index.js:118:7)
    at Parser.emit (node:events:518:28)
    at endReadableNT (node:internal/streams/readable:1696:12)
    at process.processTicksAndRejections (node:internal/process/task_queues:82:21)

Node.js v20.11.0
PS C:\Users\Ruben\Downloads\Export-To-Ghostfolio>

[dickwolff]

That's a shame. Can you maybe provide me with the input you are trying with? That way I can better reproduce the issue myself. Thanks!

[rubengr]

Transactions.csv

[dickwolff]

Thanks! I will look into it and get back to you!

[dickwolff]

I just took a look at the file you provided, but are you sure this ia a XTB export? It looks more like a DEGIRO file.

The file you provided has a lot more headers then the one in my XTB sample (which is just ID;Type;Time;Symbol;Comment;Amount).

[dickwolff]

@toffee I saw you were also looking into adding dividends to the XTB converter. Could you maybe try my fix, to verify if it all processes correctly? You can use this Docker image, or clone the branch if you want to run locally. Thanks a lot!

[toffee]

I prepared a test export. Running the code from the branch on this input file give me an error (on record 9):

# npm run start xtb

> export-to-ghostfolio@0.21.3 start
> tsx ./src/manual.ts xtb

[i] Starting Export to Ghostfolio v0.21.3
[i] Restored 0 ISIN-symbol pairs and 6 symbols from cache..
[i] Processing file using XTB converter
[i] Read CSV file. Start processing..
[i] ████████████████████░░░░░░░░░░░░░░░░░░░░ 50% | ETA 6s | Duration: 6s | 9/18/root/git/toffee/Export-To-Ghostfolio/src/converters/xtbConverter.ts:118
                let quantity = parseFloat(match[1]);
                                          ^


TypeError: Cannot read properties of null (reading '1')
    at <anonymous> (/root/git/toffee/Export-To-Ghostfolio/src/converters/xtbConverter.ts:118:43)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

Node.js v20.18.1

Additionally, seems the yahoo finance did not find correctly some stock symbols (for FLOA.UK wrongly find PFLT or for CSPX.UK find CSPX.L or for SPY5.UK find SPY508E-DC179.MX). Probably this should be discussed separately if it's an issue (or is it wrong configuration on my side!?)

[dickwolff]

Thanks for the input!

The issue seems to be with Transfer, which is not supported. I added it to the ignore list.

When I search on Yahoo Finance for FLOA.UK, I get nothing. There is a clause in the Security Service that then strips the part after the dot, so then FLOA remains. And then you find PFLT. So that makes me thinkg I that the .UK needs to be replaced with .L to be able to find the correct one.

[dickwolff]

@toffee I just pushed some changes, I also added some records of your test export. Can you verify if this works for you and all records are processed correctly?

[toffee]

With the latest changes the generation of json file ends but the content is not ok.
In my input test file the first 6 rows are: tax, dividend, tax, dividend, tax, dividend. The first record is the tax for the second record. Seems that in your code for the second record (which is dividend) you consider wrongly the tax from the third record (correct is the first record).

See bellow the generated json (3 activities) for these 6 cvs records:

   "activities": [
        {
            "accountId": "52b690dd-0133-4883-a7fb-e7464e2b92df",
            "comment": "XRAY.US USD 0.1600/ SHR",
            "fee": 3.84,
            "quantity": 16,
            "type": "DIVIDEND",
            "unitPrice": 0.16,
            "currency": "USD",
            "dataSource": "YAHOO",
            "date": "2024-07-12T13:01:17+03:00",
            "symbol": "XRAY"
        },
        {
            "accountId": "52b690dd-0133-4883-a7fb-e7464e2b92df",
            "comment": "XRAY.US USD 0.1600/ SHR",
            "fee": 1.44,
            "quantity": 80,
            "type": "DIVIDEND",
            "unitPrice": 0.16,
            "currency": "USD",
            "dataSource": "YAHOO",
            "date": "2024-07-12T13:01:17+03:00",
            "symbol": "XRAY"
        },
        {
            "accountId": "52b690dd-0133-4883-a7fb-e7464e2b92df",
            "comment": "XRAY.US USD 0.1600/ SHR",
            "fee": 0,
            "quantity": null,
            "type": "DIVIDEND",
            "unitPrice": null,
            "currency": "USD",
            "dataSource": "YAHOO",
            "date": "2024-07-12T13:01:17+03:00",
            "symbol": "XRAY"
        },

As you can see, for the first two activities the fee is wrong and for the third activity has quantity and unitPrice null.

To change the UK to L don't think it's correct - the symbol as it's come from XTB is correct (except the ones with US). I guess the removal of ".country" should be done only for US (in case of yahoo finance).

[dickwolff]

I will look into the tax issue.

But when I remove the .UK (as previoulsy), then you don't get completely different symbols right? The symbols that XTB uses are not known in Yahoo Finance. So with this replacement of UK by L you do at least get the security you bought at XTB?

[toffee]

Thank you. My previous assumption was wrong. I'm getting the correct security with your change from UK to L.

[rubengr]

@dickwolff you are completely right. Sorry for my mistake.
I get same error with file
xStation5_cashOperations.csv

[i] Starting Export to Ghostfolio v0.22.1
[i] Restored 7 ISIN-symbol pairs and 13 symbols from cache..
[i] Processing file using XTB converter
[i] Read CSV file. Start processing..
[i] █░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ 2% | ETA 609s | Duration: 16s | 17/765file:///C:/Users/Ruben/Downloads/Exp1
var __defProp=Object.defineProperty;var __name=(target,value)=>__defProp(target,"name",{value,configurable:true});impor.


TypeError: Cannot read properties of null (reading '1')
    at <anonymous> (c:\Users\Ruben\Downloads\Export-To-Ghostfolio\src\converters\xtbConverter.ts:122:43)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

Node.js v20.11.0

[dickwolff]

No problem @rubengr. I will investigate further and let you know what i find!