Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[16.0][BIG] Large refactoring/improvement/cleanup of OCA/bank-payment #1174

Open
wants to merge 33 commits into
base: 16.0
Choose a base branch
from

Conversation

alexis-via
Copy link
Contributor

@alexis-via alexis-via commented Nov 7, 2023

NEW FEATURES

  • activate 'in_payment' payment_state on invoices... at last !
  • payment order numbering: add ability to have a sequence specific to a particular payment mode (optional)
  • add show account number policy "first_last" to show N first and N last characters of bank account number
  • add support for regulatory reporting
  • replace unstructured address by structured address in SEPA XML file (mandatory starting 11/2025)
  • add support for pain.008.01.08 (SDD) and pain.001.001.09 (SCT), which are now the recommended versions of the EPC

SMALL IMPROVEMENTS

  • account.payment.method: rename field "payment_order_only" to "payment_order_ok" for naming consistency with the field of account.payment.mode. Auto-set "payment_order_ok" of payment mode from the equivalent field of payment method. Migration scripts provided for account.payment.method and the data of SEPA credit transfer and SEPA direct debit.
  • payment order attachment: add M2O payment_file_id that points to the payment file. No more need for a simplified attachment form view.
  • keep the spaces in the scrambled bank account number, so that the user easily understands that it is a "scrambled" bank account number
  • add field acc_number_scrambled on res.partner.bank for easy and direct use of scrambled account number
  • improve the layout of the sepa section of the accounting config page
  • hide field mandate_id on form + tree views of payment lines if payment mode is mandate_required=False
  • automatically open download pop-up when clicking on "Generate payment file".
  • set no_debit_before_maturity to True by default on payment modes
  • set default_date_prefered to "due" by default on inbound payment modes with payment_order_ok = True

SIMPLIFICATION / CODE CLEANUP

  • big cleanup of XML code generation: use lxml.objectify instead of lxml.etree, add support for currencies with decimal places != 2, stop using safe_eval()
  • remove field partner_bank_id on account.move.line
  • move code for scrambled account number from qweb report to res.partner.bank class
  • remove boolean field show_bank_account_from_journal from account.payment.mode: we now consider that this option is always true i.e. when account_payment_partner is installed, Odoo takes the bank accounts to display on the customer invoice from the payment mode (if the payment mode is empty, it displays all the bank accounts of the company). As a consequence, hide partner_bank_id on customer invoice form view.
  • remove fields inbound_payment_order_only and outbound_payment_order_only from account.journal : these 2 fields were not used!
  • add check_company=True where it was missing and remove code of constraint that checks company consistency
  • re-organise form view of payment order to equilibrate between left column and right column
  • remove support for pain.001.001.02/04/05 (SCT) and pain.008.01.03/04 (SDD) which have never been selected by the EPC

TECHNICAL:

  • replace @api.onchange by computed readonly=False store=True fields
  • remove (or reduce) the use of demo data in tests
  • speed-up tests by using tracking_disable=True
  • Replace (6, 0, []) and (0, 0, {}) by Command.set() and Command.create()
  • add sql unicity constraint on payment order number per company

FIXES:

NEW FEATURES
- activate 'in_payment' payment_state on invoices... at last !
- payment order numbering: add ability to have a sequence specific to a particular payment mode (optional)
- add show account number policy "first_last" to show N first and N last characters of bank account number

SMALL IMPROVEMENTS
- account.payment.method: rename field "payment_order_only" to "payment_order_ok" for naming consistency with the field of account.payment.mode. Auto-set "payment_order_ok" of payment mode from the equivalent field of payment method. Migration scripts provided for account.payment.method and the data of SEPA credit transfer and SEPA direct debit.
- payment order attachment: add M2O payment_file_id that points to the payment file. No more need for a simplified attachment form view.
- keep the spaces in the scrambled bank account number, so that the user easily understands that it is a "scrambled" bank account number
- add field acc_number_scrambled on res.partner.bank for easy and direct use of scrambled account number

SIMPLIFICATION / CODE CLEANUP
- remove field partner_bank_id on account.move.line
- move code for scrambled account number from qweb report to res.partner.bank class
- remove boolean field show_bank_account_from_journal from account.payment.mode: we now consider that this option is always true i.e. when account_payment_partner is installed, Odoo takes the bank accounts to display on the customer invoice from the payment mode (if the payment mode is empty, it displays all the bank accounts of the company). As a consequence, hide partner_bank_id on customer invoice form view.
- remove fields inbound_payment_order_only and outbound_payment_order_only from account.journal : these 2 fields were not used!
- add check_company=True where it was missing and remove code of constraint that checks company consistency
- re-organise form view of payment order to equilibrate between left column and right column

TECHNICAL:
- replace @api.onchange by computed readonly=False store=True fields
- remove (or reduce) the use of demo data in tests
- speed-up tests by using tracking_disable=True
- Replace (6, 0, []) and (0, 0, {}) by Command.set() and Command.create()
- add sql unicity constraint on payment order number per company
@alexis-via
Copy link
Contributor Author

It's the module account_payment_order_grouped_output which make the tests for payment_status in_payment vs paid fail. I have never used account_payment_order_grouped_output ; according to README, it has been developed to mitigate the drawbacks of the switch to account.payment !!!

@alexis-via
Copy link
Contributor Author

For some strange reasons that I don't understand, when you upgrade to this branch, Odoo fails because a view has been deleted in the source code. You can solve the issue by executing this SQL request:

delete from ir_ui_view where id=(select res_id from ir_model_data where module='account_payment_order' and model='ir.ui.view' and name='view_move_line_form');

@alexis-via alexis-via force-pushed the 16-payment_order-improve_and_cleanup branch from 4584b38 to 81a7c2f Compare November 20, 2023 15:43
@alexis-via
Copy link
Contributor Author

Tests are not green.

@alexis-via
Copy link
Contributor Author

Payment order, in_payment and bank statement reconciliation

In this PR, I activate the "in_payment" payment status on invoice. For me, it's one of the major advantages of using the object account.payment, that was implemeted by @pedrobaeza in this big PR #979
Surprisingly, Pedro didn't activate the in_payment status in his PR. In fact, activating such a feature is very easy, you just have to inherit the method _get_invoice_in_payment_state() in account.move and return "in_payment" instead of "paid":
https://github.com/odoo/odoo/blob/16.0/addons/account/models/account_move.py#L4345
Only 1 line of code!

One of the drawbacks in the switch to account.payment is that, when you validate a payment order with 100 different partners, you get 100 different entries in the Outstanding receipts account. So, when you have the bank statement line with the total amount of the payment order, you have to select 100 counter-part entries in the Outstanding receipts account !!!

The module account_payment_order_grouped_output is designed to solve this problem:
https://github.com/OCA/bank-payment/tree/16.0/account_payment_order_grouped_output
It generates an additional account.move:

  • 100 lines in the Outstanding receipts account with reconciliation
  • 1 counter part line for the total amount... in the Outstanding receipts account
    Advantage: in the bank statement reconcile interface, you just have to select 1 counterpart line.
    Drawbacks: it breaks the "in_payment" payment status, because the counter-part line of the account.move generated by the account.payment is reconciled when the payment order is validated and not when the bank statement line is processed ! Looking at the code about this in the "account" module, it's not easy to modify that.

I was told that, in the Enterprise module, they have a special tab in the bank statement reconcile interface for payments by lot: so, when you process the bank statement lines with the total amount of the payment order, Odoo proposes the payment orders not yet reconciled with the bank statement lines ; when you select the appropriate payment order, it will automatically set the 100 counterpart lines to reconcile the bank statement line.
It seems that the OCA module account_reconcile_payment_order
https://github.com/OCA/account-reconcile/tree/15.0/account_reconcile_payment_order
is designed to propose a similar feature in the community bank statement reconcile interface. But this module is not available in Odoo v16 with the new account_reconcile_oca module.

So, how should be move forward on this issue ? @pedrobaeza @etobella

My opinion:

  1. account_payment_order_grouped_output is not a good option: we would have to modify the native behavior of the computation of the in_payment state, which is not easy and not desirable. In addition, it adds an extra account.move.
  2. a solution similar to Odoo Enterprise seemed a bit strange to me at first... but, after some discussions with other community member, I eventually think it may be a good solution. If account_reconcile_payment_order is designed for that, we could port it to v16 and make it depend on account_reconcile_oca. It's probably not easy, but it's really necessary if we want to have both the "in_payment" payment_status on invoices and an easy reconcile process in the bank statement, even for payment orders with a lot of different partners.

@pedrobaeza
Copy link
Member

Yes, account_reconcile_payment_order adapted to the new reconcilation system sees the way to go. This module can add a new tab "Payment/Debit orders" that allows to select the non-reconciled ones, and clicking on one of them, it adds all the AR/AP generated lines (but allowing to remove/re-add some of them if needed).

Harmonize values for reference_type on account.move and communication_type on account.payment.line:
2 possible values : free and structured (migration script provided)
Simplify code for reference.
Copy link
Contributor

@miquelalzanillas miquelalzanillas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What a great PR!

Tested functionallity in a fresh new db and seems is working.

@etobella
Copy link
Member

@alexis-via Can you rebase?

@@ -657,7 +654,7 @@ def generate_remittance_info_block(self, parent_node, line, gen_args):
creditor_ref_info_type_issuer = etree.SubElement(
creditor_ref_info_type, "Issr"
)
creditor_ref_info_type_issuer.text = communication_type
creditor_ref_info_type_issuer.text = "ISO"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@alexis-via please keep communication_type. We need it for belgium where (IIRC) it must become bba.
https://github.com/OCA/l10n-belgium/tree/10.0/l10n_be_iso20022_pain

cc/ @luc-demeyer

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sbidoul
I fully agree.
Small remark on l10n_be_iso20022_pain: should be 'BBA' (uppercase), cf. my comment on 16.0 PR.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for pointing this out. Let's talk about this during the next code sprint, because the datamodel for structured communication type is really messy and unclean. I'm confident that we'll agree on a nice and clear datamodel on this, that will make the l10n_be_iso20022_pain simpler and make the XML generation code simpler.

@luc-demeyer
Copy link
Contributor

@alexis-via
I am still waiting on merge of PR's we made during OCA days and rebase of this PR before I start in-depth code review and testing of this PR.

@Mat-moran
Copy link

Mat-moran commented Dec 18, 2023

Hi @luc-demeyer , which PR are you waiting for? Can we help to push this? Is blocking migration to v17

@pedrobaeza
Copy link
Member

Please rebase and solve conflicts.

@alexis-via
Copy link
Contributor Author

I'll continue to work on this next month (but not in the coming days)

@pedrobaeza pedrobaeza added this to the 16.0 milestone Feb 9, 2024
Copy link
Member

@pedrobaeza pedrobaeza left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Half of the files reviewed.

res.append((rec.id, "[%s] %s" % (rec.code, rec.name)))
return res

def _name_search(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of this, you can just put _rec_names_search = ["code", "name"].

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. Thanks for your remark!

@@ -0,0 +1,3 @@
id,name,model_id:id,group_id:id,perm_read,perm_write,perm_create,perm_unlink
access_account_pain_regulatory_reporting_read,Read access on account.pain.regulatory.reporting,model_account_pain_regulatory_reporting,base.group_user,1,0,0,0
access_account_pain_regulatory_reporting_full,Full access on account.pain.regulatory.reporting,model_account_pain_regulatory_reporting,account.group_account_manager,1,1,1,1
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't be the full right access the "Payment" group?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

<field name="country_id" />
<field name="active" invisible="1" />
</group>
</form>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Incorrect indentation

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I let prettier do the formatting work and pre-commit doesn't give any error. Strange !

@openupgrade.migrate()
def migrate(env, version):
sct_method = env.ref("account_banking_sepa_credit_transfer.sepa_credit_transfer")
sct_method.write({"payment_order_ok": True})
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't force this. Just rename the column of the field on the source one.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But the payment method for SEPA credit transfer should always have payment_order_ok = true, no ? Otherwise, what's the point of installing account_banking_sepa_credit_transfer ??

if line_purpose:
purpose = etree.SubElement(credit_transfer_transaction_info, "Purp")
etree.SubElement(purpose, "Cd").text = line_purpose
payment_line = line.payment_line_ids[0]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why changing this? Are you sure there's always going to be a payment_line_id?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's just the same code that was moved from account_banking_sepa_credit_transfer to account_banking_pain_base, to mutualize the code with account_banking_sepa_direct_debit. On a payment order, an account.payment is always linked to N account.payment.line, and we read the purpose on the account.payment.line because that's where the information is. I didn't change the logic.

data = {
"partner_id": partner_id,
"reference_type": "none",
"reference_type": "free",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do you need this change?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I changed the value in the module account_payment_order to have the same value ("free") on account.move and account.payment.line for unstructured communication. Avoids a headache for maintainers. Migration scripts are provided in account_payment_order, cf https://github.com/akretion/banking/blob/16-payment_order-improve_and_cleanup/account_payment_order/migrations/16.0.2.0.0/pre-migration.py#L14

@openupgrade.migrate()
def migrate(env, version):
sct_method = env.ref("account_banking_sepa_direct_debit.sepa_direct_debit")
sct_method.write({"payment_order_ok": True})
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The same about not forcing

@rousseldenis
Copy link
Contributor

@alexis-via Any news on this ?

@alexis-via
Copy link
Contributor Author

My re-write of the code that generate the address block fixes the bug reported here:
#1209
I saw the bug when I was working on that code... I just wasn't aware someone had reported it !

@alexis-via
Copy link
Contributor Author

While reading the code, I found a new bug on direct debit, that I later confirmed through tests. I opened a bug report #1336 and I fixed it in my last commit.

…to filter partner_bank_id on account.move

Use native method is_inbound() and is_outbound()
Only copy bank account from move line to payment line for payment
orders, not for debit orders.
… linked to debit orders that don't have mandate_required = True
On payment method, check that mandate_required is only be True on
inbound methods.
…account.payment.line (same as in account_payment_order)
… to an IBAN account

Add python constraint to disallow the creation of a SEPA mandate linked
to a non-IBAN account. Fixes bug OCA#1337
@flotho
Copy link
Member

flotho commented Aug 25, 2024

Hi @alexis-via
Thanks for this PR. I still have an issue when generating the sepa file . All the account are set to the wrong value.
I've tested this with the latest changes of your PRs
I made a proposal here #1212

Regards

When the user clicks on the button "Generate payment file" in the form
view of payment order, it gets a pop-up to save the file.
…nt.payment.order

This new field payment_method_code on account.move and
account.payment.order is available as invisible in tree+form view: it
allows to easily show/hide fields specific to a payment method in format-specific modules.
Remove field sepa_payment_method (replaced by generic field payment_method_code)
Update field definition of payment_mode_id on account.move: precompute=True and move domain from view to field definition
@MiquelRForgeFlow
Copy link
Contributor

It's too late for v16.

We still have few clients in v12. Never it's too late.

If we use a payment order with mandate required, mandate is placed
before bank account (more logic) and bank account becomes readonly: it
is set by the mandate.
@alexis-via
Copy link
Contributor Author

Yesterday, I added a small but nice improvement : automatically open download pop-up when clicking on "Generate payment file".

@alexis-via
Copy link
Contributor Author

I made 2 other minors changes to have "good practise" by default when configuring payment modes:

  • set no_debit_before_maturity to True by default on payment modes
  • set default_date_prefered to "due" by default on inbound payment modes with payment_order_ok = True

@flotho
Copy link
Member

flotho commented Sep 3, 2024

Hi @alexis-via ,

Any chance to cherry pick this commit flotho@da2028c to FIX #1193
and FIX #1131

Moreover, I faced this issue after an upgrade of the pain base module while trying to generate a file :

image

Any tips to find which state field is concerned ?

Regards

@flotho
Copy link
Member

flotho commented Sep 3, 2024

Hi @alexis-via ,
.../...
Moreover, I faced this issue after an upgrade of the pain base module while trying to generate a file :

image

Any tips to find which state field is concerned ?

Regards

Found, state_id was missing on the partner

@flotho
Copy link
Member

flotho commented Oct 7, 2024

Hi @alexis-via , any chance to have a rebase here ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.