CGM_Public/pretix_cgo
1
0
Fork 0
forked from CGM_Public/pretix_original
Code Pull Requests Actions Activity

Banktransfer: Handle trailing commas in headers for Lloyds Bank CSV files (#4782)

Browse Source 
Lloyds Bank (UK) CSV files include a trailing comma in the header row
but not in the data rows, causing the `csvimport.parse` function to
skip the data rows. This occurs because the header length exceeds the
row length, making them unequal to `hint.cols`.

This commit adjusts the length check to allow a range of acceptable row
lengths, from the index of the last non-empty column in the header to
`hint.cols`. This ensures compatibility with headers containing one or
more trailing commas without affecting rows with correctly labelled columns.

The solution avoids breaking changes by leaving underlying data structures
untouched. Alternative approaches, such as dropping trailing commas before
parsing or removing empty elements after parsing, were avoided due to
potential risks. Specifically, trailing columns might contain data that
banks provide but fail to label in the header row.
This commit is contained in:
Kian Cross
2025-02-05 15:56:28 +00:00
committed by GitHub
parent 03d3879787
commit 5d4b218aa6
3 changed files with 42 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
src/tests/plugins/banktransfer/csvimport_data_gb_lloyds.csv Normal file
View File

@@ -0,0 +1,3 @@
Transaction Date,Transaction Type,Sort Code,Account Number,Transaction Description,Debit Amount,Credit Amount,Balance,
27/01/2025,FPI,'99-99-99,11111111,SMITH J ABCDE 111111111111111111 111111 10 25JAN25 13:34,,214,500
25/01/2025,FPI,'99-99-99,11111111,JONES A FGHIJ 111111111111111112 111112 10 25JAN25 10:34,,213,286
1 Transaction Date,Transaction Type,Sort Code,Account Number,Transaction Description,Debit Amount,Credit Amount,Balance,
2 27/01/2025,FPI,'99-99-99,11111111,SMITH J ABCDE 111111111111111111 111111 10 25JAN25 13:34,,214,500
3 25/01/2025,FPI,'99-99-99,11111111,JONES A FGHIJ 111111111111111112 111112 10 25JAN25 10:34,,213,286

25
src/tests/plugins/banktransfer/test_csvparser.py
View File

@@ -166,3 +166,28 @@ class CsvImportTest(TestCase):
]
filename = "csvimport_data_de_postbank.csv"
self._test_from_sample_file(filename, expected, hint, expected_parsed)
def test_sample_file_lloyds(self):
expected = [
# Lloyds Bank includes a trailing comma at the end of the header row, making the header one column longer than the data rows.
['Transaction Date', 'Transaction Type', 'Sort Code', 'Account Number', 'Transaction Description', 'Debit Amount',
'Credit Amount', 'Balance', ''],
["27/01/2025", "FPI", "'99-99-99", "11111111", "SMITH J ABCDE 111111111111111111 111111 10 25JAN25 13:34",
"", "214", "500"],
["25/01/2025", "FPI", "'99-99-99", "11111111", "JONES A FGHIJ 111111111111111112 111112 10 25JAN25 10:34",
"", "213", "286"],
]
hint = {
'reference': [4],
'date': 0,
'amount': 6,
'cols': 9,
}
expected_parsed = [
{'reference': 'SMITH J ABCDE 111111111111111111 111111 10 25JAN25 13:34', 'amount': '214',
'date': '27/01/2025'},
{'reference': 'JONES A FGHIJ 111111111111111112 111112 10 25JAN25 10:34', 'amount': '213',
'date': '25/01/2025'},
]
filename = "csvimport_data_gb_lloyds.csv"
self._test_from_sample_file(filename, expected, hint, expected_parsed)