Refactor batch payer

[rvermootenct]

---

name: Pull Request
about: Create a pull request to make a contribution
labels:

---

IMPORTANT NOTICE:
I read and understood the guidelines for contributions to the TRD. The contribution may qualify for being compensated by the TRD grant if approved by the maintainers.

Please be aware that the failing tests can be seen on master as well atm..

This PR resolves the issue 641. The following steps were performed:

• Analysis: The batch payer class is very tough to work with and has no unit tests. It requires a huge refactor but needs initial building blocks before that will be possible. There is a lot of technical debt in this class that needs to be paid.

• Solution: BatchPayer needs to have functions pulled out and unit tested. Naming conventions need to be better.

• Implementation: Pulled out a bunch of little functions into a utils file. Renamed a number of vars within the class for better understanding. Wrote tests.

• Performed tests: Tests should pass no problem and I was able to test locally on ghostnet that the batch payer acted as expected.

Work effort: 20h

GIF TAX:

[jdsika]

I merged #623 in order to have a solution for the release.

[jdsika]

LGTM, I will let @dansan566 make 1 payment with this branch and merge afterwards

[dansan566]

Did a dry run, and got this:

Error, response ->Failed to parse the request body: Malformed value<-
2023-01-17 17:22:19,980 - consumer0 - ERROR - Error in run_operation
2023-01-17 17:22:19,980 - consumer0 - INFO - Payment to KT1XGnEMZYXJWKgwNCctGNzPYT7hXVSSMABe script could not be processed. Possible reason: liquidated contract. Skipping. Think about redirecting the payout to the owner address using the maps rules. Please refer to the TRD documentation or to one of the TRD maintainers.
2023-01-17 17:22:19,981 - consumer0 - ERROR - Error, request ->POST http://127.0.0.1:8732/chains/main/blocks/head/helpers/scripts/run_operation<-, params ->None<-,

Payouts to the address worked on the main branch, and the contract is not liquidated!
https://tzkt.io/KT1XGnEMZYXJWKgwNCctGNzPYT7hXVSSMABe/operations/

I had this error on more than 30 addresses, all of them KT1... (smart contracts)

[jdsika]

@dansan566 ok thank you for the insight! retry failed payments on master and we will investigate

[dansan566]

@jdsika, there are no failed payments if skipped!

[jdsika]

Did a dry run, and got this:

Error, response ->Failed to parse the request body: Malformed value<-
2023-01-17 17:22:19,980 - consumer0 - ERROR - Error in run_operation
2023-01-17 17:22:19,980 - consumer0 - INFO - Payment to KT1XGnEMZYXJWKgwNCctGNzPYT7hXVSSMABe script could not be processed. Possible reason: liquidated contract. Skipping. Think about redirecting the payout to the owner address using the maps rules. Please refer to the TRD documentation or to one of the TRD maintainers.
2023-01-17 17:22:19,981 - consumer0 - ERROR - Error, request ->POST http://127.0.0.1:8732/chains/main/blocks/head/helpers/scripts/run_operation<-, params ->None<-,

Payouts to the address worked on the main branch, and the contract is not liquidated! https://tzkt.io/KT1XGnEMZYXJWKgwNCctGNzPYT7hXVSSMABe/operations/

I had this error on more than 30 addresses, all of them KT1... (smart contracts)

Two issues.

• First it should be FAILED instead of SKIPPED
• Parsing should be working

[rvermootenct]

Diff of calculations on dry run mainnet between master and this branch:

Diff of payments on dry run mainnet between master and this branch

NOTE: On the payments there is a diff but it is only the position of the entry, the entry itself is identical.

[dansan566]

@jdsika I did a dry run on the branch, and it worked so the error has been fixed by @rvermootenct
The next payout will be on this one!
My help with debugging: 1.5h

[jdsika]

Diff of calculations on dry run mainnet between master and this branch:

Diff of payments on dry run mainnet between master and this branch

NOTE: On the payments there is a diff but it is only the position of the entry, the entry itself is identical.

Please create a follow up ticket why the heck those sortings are non-deterministic. This is killing my test engineering soul

[nicolasochem]

🔥 🔥

[nicolasochem]

You know what else is "unmanageable" (per #641)?
The producer/consumer pattern.
See points made in #491

TRD would be so much simpler if it was doing what it needs to do in order, without a central "bus" of tasks.