Improve dialog

Co-authored-by: Raphael Michel <michel@rami.io>

.github/dependabot.yml

@@ -0,0 +1,15 @@
+# To get started with Dependabot version updates, you'll need to specify which
+# package ecosystems to update and where the package manifests are located.
+# Please see the documentation for all configuration options:
+# https://help.github.com/github/administering-a-repository/configuration-options-for-dependency-updates
+
+version: 2
+updates:
+  - package-ecosystem: "pip"
+    directory: "/src"
+    schedule:
+      interval: "daily"
+  - package-ecosystem: "npm"
+    directory: "/src/pretix/static/npm_dir"
+    schedule:
+      interval: "monthly"

doc/admin/errors.rst

@@ -0,0 +1,40 @@
+.. _`admin-errors`:
+
+Dealing with errors
+===================
+
+If you encounter an error in pretix, please follow the following steps to debug it:
+
+* If the error message is shown on a **white page** and the last line of the error includes "nginx", the error is not with pretix
+  directly but with your nginx webserver. This might mean that pretix is not running, but it could also be something else.
+  Please first check your nginx error log. The default location is ``/var/log/nginx/error.log``.
+
+  * If it turns out pretix is not running, check the output of ``docker logs pretix`` for a docker installation and
+    ``journalctl -u pretix-web.service`` for a manual installation.
+
+* If the error message is an "**Internal Server Error**" in purple pretix design, please check pretix' log file which by default is at
+  ``/var/pretix-data/logs/pretix.log`` if you installed with docker and ``/var/pretix/data/logs/pretix.log`` otherwise. If you don't
+  know how to interpret it, open a discussion on GitHub with the relevant parts of the log file.
+
+  * If the error message includes ``/usr/bin/env: ‘node’: No such file or directory``, you forgot to install ``node.js``
+
+  * If the error message includes ``OfflineGenerationError``, you might have forgot to run the ``rebuild`` step after a pretix update
+    or plugin installation.
+
+  * If the error message mentions your database server or redis server, make sure these are running and accessible.
+
+* If pretix loads fine but certain actions (creating carts, orders, or exports, downloading tickets, sending emails) **take forever**,
+  ``pretix-worker`` is not running. Check the output of ``docker logs pretix`` for a docker installation and
+  ``journalctl -u pretix-worker.service`` for a manual installation.
+
+* If the page loads but all **styles are missing**, you probably forgot to update your nginx configuration file after an upgrade of your
+  operating system's python version.
+
+
+If you are unable to debug the issue any further, please open a **discussion** on GitHub in our `Q&A Forum`_. Do **not** open an issue
+right away, since most things turn out not to be a bug in pretix but a mistake in your server configuration. Make sure to include
+relevant log excerpts in your question.
+
+If you're a pretix Enterprise customer, you can also reach out to support@pretix.eu with your issue right away.
+
+.. _Q&A Forum: https://github.com/pretix/pretix/discussions/categories/q-a

doc/admin/index.rst

@@ -9,7 +9,9 @@ This documentation is for everyone who wants to install pretix on a server.
    :maxdepth: 2
 
    installation/index
+   updates
    config
    maintainance
    scaling
+   errors
    indexes

doc/admin/indexes.rst

@@ -50,7 +50,7 @@ Here is the currently recommended set of commands::
     CREATE INDEX CONCURRENTLY pretix_addidx_orderpos_name
         ON pretixbase_orderposition
         USING gin (upper("attendee_name_cached") gin_trgm_ops);
-    CREATE INDEX CONCURRENTLY pretix_addidx_orderpos_scret
+    CREATE INDEX CONCURRENTLY pretix_addidx_orderpos_secret
         ON pretixbase_orderposition
         USING gin (upper("secret") gin_trgm_ops);
     CREATE INDEX CONCURRENTLY pretix_addidx_orderpos_email

doc/admin/installation/docker_smallscale.rst

@@ -256,6 +256,8 @@ create an event and start selling tickets!
 
 You should probably read :ref:`maintainance` next.
 
+.. _`docker_updates`:
+
 Updates
 -------
 
@@ -271,6 +273,8 @@ Restarting the service can take a few seconds, especially if the update requires
 Replace ``stable`` above with a specific version number like ``1.0`` or with ``latest`` for the development
 version, if you want to.
 
+Make sure to also read :ref:`update_notes` and the release notes of the version you are updating to.
+
 .. _`docker_plugininstall`:
 
 Install a plugin

doc/admin/installation/manual_smallscale.rst

@@ -280,6 +280,8 @@ create an event and start selling tickets!
 
 You should probably read :ref:`maintainance` next.
 
+.. _`manual_updates`:
+
 Updates
 -------
 
@@ -294,6 +296,7 @@ To upgrade to a new pretix release, pull the latest code changes and run the fol
     (venv)$ python -m pretix updatestyles
     # systemctl restart pretix-web pretix-worker
 
+Make sure to also read :ref:`update_notes` and the release notes of the version you are updating to.
 
 .. _`manual_plugininstall`:
 

doc/admin/maintainance.rst

@@ -9,6 +9,8 @@ If you host your own pretix instance, you also need to care about the availabili
 of your service and the safety of your data yourself. This page gives you some
 information that you might need to do so properly.
 
+.. _`backups`:
+
 Backups
 -------
 

doc/admin/updates.rst

@@ -0,0 +1,51 @@
+.. _`update_notes`:
+
+Update notes
+============
+
+pretix receives regular feature and bugfix updates and we highly encourage you to always update to
+the latest version for maximum quality and security. Updates are announces on our `blog`_. There are
+usually 10 feature updates in a year, so you can expect a new release almost every month.
+
+Pure bugfix releases are only issued in case of very critical bugs or security vulnerabilities. In these
+case, we'll publish bugfix releases for the last three stable release branches.
+
+Compatibility to plugins and in very rare cases API clients may break. For in-depth details on the
+API changes of every version, please refer to the release notes published on our blog.
+
+Upgrade steps
+-------------
+
+For the actual upgrade, you can usually just follow the steps from the installation guide for :ref:`manual installations <manual_updates>`
+or :ref:`docker installations <docker_updates>` respectively.
+Generally, it is always strongly recommended to perform a :ref:`backup <backups>` first.
+It is possible to skip versions during updates, although we recommend not skipping over major version numbers
+(i.e. if you want to go from 2.4 to 4.4, first upgrade to 3.0, then upgrade to 4.0, then to 4.4).
+
+In addition to these standard update steps, the following list issues steps that should be taken when you upgrade
+to specific versions for pretix. If you're skipping versions, please read the instructions for every version in
+between as well.
+
+Upgrade to 3.17.0 or newer
+""""""""""""""""""""""""""
+
+pretix 3.17 introduces a dependency on ``nodejs``, so you should install it on your system::
+
+    # apt install nodejs npm
+
+Upgrade to 4.4.0 or newer
+"""""""""""""""""""""""""
+
+pretix 4.4 introduces a new data structure to store historical financial data. If you already have existing
+data in your database, you will need to back-fill this data or you might get incorrect reports! This is not
+done automatically as part of the usual update steps since it can take a while on large databases and you might
+want to do it in parallel while the system is already running again. Please execute the following command::
+
+    (venv)$ python -m pretix create_order_transactions
+
+Or, with a docker installation::
+
+    $ docker exec -it pretix.service pretix create_order_transactions
+
+
+.. _blog: https://pretix.eu/about/en/blog/

doc/api/resources/index.rst

@@ -31,5 +31,6 @@ Resources and endpoints
    webhooks
    seatingplans
    exporters
+   sendmail_rules
    billing_invoices
    billing_var

doc/api/resources/invoices.rst

@@ -78,6 +78,12 @@ lines                                 list of objects            The actual invo
                                                                  an event series not created by a product (e.g. shipping or
                                                                  cancellation fees) as well as whenever the respective (sub)event
                                                                  has no end date set.
+├ event_location                      string                     Location of the (sub)event this line was created for as it
+                                                                 was set during invoice creation. Can be ``null`` for all invoice
+                                                                 lines created before this was introduced as well as for lines in
+                                                                 an event series not created by a product (e.g. shipping or
+                                                                 cancellation fees) as well as whenever the respective (sub)event
+                                                                 has no location set.
 ├ attendee_name                       string                     Attendee name at time of invoice creation. Can be ``null`` if no
                                                                  name was set or if names are configured to not be added to invoices.
 ├ gross_value                         money (string)             Price including taxes
@@ -110,6 +116,10 @@ internal_reference                    string                     Customer's refe
 
    The attributes ``fee_type`` and ``fee_internal_type`` have been added.
 
+.. versionchanged:: 4.1
+
+   The attribute ``lines.event_location`` has been added.
+
 
 Endpoints
 ---------
@@ -179,6 +189,7 @@ Endpoints
                 "fee_internal_type": null,
                 "event_date_from": "2017-12-27T10:00:00Z",
                 "event_date_to": null,
+                "event_location": "Heidelberg",
                 "attendee_name": null,
                 "gross_value": "23.00",
                 "tax_value": "0.00",
@@ -267,6 +278,7 @@ Endpoints
             "fee_internal_type": null,
             "event_date_from": "2017-12-27T10:00:00Z",
             "event_date_to": null,
+            "event_location": "Heidelberg",
             "attendee_name": null,
             "gross_value": "23.00",
             "tax_value": "0.00",

doc/api/resources/item_variations.rst

@@ -25,6 +25,8 @@ description                           multi-lingual string       A public descri
                                                                  Markdown syntax or can be ``null``.
 position                              integer                    An integer, used for sorting
 require_membership                    boolean                    If ``true``, booking this variation requires an active membership.
+require_membership_hidden             boolean                    If ``true`` and ``require_membership`` is set, this variation will
+                                                                 be hidden from users without a valid membership.
 require_membership_types              list of integers           Internal IDs of membership types valid if ``require_membership`` is ``true``
 sales_channels                        list of strings            Sales channels this variation is available on, such as
                                                                  ``"web"`` or ``"resellers"``. Defaults to all existing sales channels.
@@ -75,6 +77,7 @@ Endpoints
             },
             "active": true,
             "require_membership": false,
+            "require_membership_hidden": false,
             "require_membership_types": [],
             "sales_channels": ["web"],
             "available_from": null,
@@ -95,6 +98,7 @@ Endpoints
             },
             "active": true,
             "require_membership": false,
+            "require_membership_hidden": false,
             "require_membership_types": [],
             "description": {},
             "position": 1,
@@ -144,6 +148,7 @@ Endpoints
         "original_price": null,
         "active": true,
         "require_membership": false,
+        "require_membership_hidden": false,
         "require_membership_types": [],
         "sales_channels": ["web"],
         "available_from": null,
@@ -179,6 +184,7 @@ Endpoints
         "default_price": "10.00",
         "active": true,
         "require_membership": false,
+        "require_membership_hidden": false,
         "require_membership_types": [],
         "sales_channels": ["web"],
         "available_from": null,
@@ -204,6 +210,7 @@ Endpoints
         "original_price": null,
         "active": true,
         "require_membership": false,
+        "require_membership_hidden": false,
         "require_membership_types": [],
         "sales_channels": ["web"],
         "available_from": null,
@@ -260,6 +267,7 @@ Endpoints
         "original_price": null,
         "active": false,
         "require_membership": false,
+        "require_membership_hidden": false,
         "require_membership_types": [],
         "sales_channels": ["web"],
         "available_from": null,

doc/api/resources/items.rst

@@ -70,6 +70,8 @@ require_approval                      boolean                    If ``true``, or
                                                                  paid.
 require_bundling                      boolean                    If ``true``, this item is only available as part of bundles.
 require_membership                    boolean                    If ``true``, booking this item requires an active membership.
+require_membership_hidden             boolean                    If ``true`` and ``require_membership`` is set, this product will
+                                                                 be hidden from users without a valid membership.
 require_membership_types              list of integers           Internal IDs of membership types valid if ``require_membership`` is ``true``
 grant_membership_type                 integer                    If set to the internal ID of a membership type, purchasing this item will
                                                                  create a membership of the given type.
@@ -105,6 +107,8 @@ variations                            list of objects            A list with one
 ├ active                              boolean                    If ``false``, this variation will not be sold or shown.
 ├ description                         multi-lingual string       A public description of the variation. May contain
 ├ require_membership                  boolean                    If ``true``, booking this variation requires an active membership.
+├ require_membership_hidden           boolean                    If ``true`` and ``require_membership`` is set, this variation will
+                                                                 be hidden from users without a valid membership.
 ├ require_membership_types            list of integers           Internal IDs of membership types valid if ``require_membership`` is ``true``
                                                                  Markdown syntax or can be ``null``.
 ├ sales_channels                      list of strings            Sales channels this variation is available on, such as
@@ -155,6 +159,10 @@ meta_data                             object                     Values set for
    The attributes ``require_membership``, ``require_membership_types``, ``grant_membership_type``, ``grant_membership_duration_like_event``,
     ``grant_membership_duration_days`` and ``grant_membership_duration_months`` have been added.
 
+.. versionchanged:: 4.4
+
+   The attributes ``require_membership_hidden`` attribute has been added.
+
 Notes
 -----
 

doc/api/resources/orders.rst

@@ -128,6 +128,10 @@ last_modified                         datetime                   Last modificati
 
    The ``custom_followup_at`` attribute has been added.
 
+.. versionchanged:: 4.4
+
+   The ``item`` and ``variation`` query parameters have been added.
+
 
 .. _order-position-resource:
 
@@ -415,6 +419,8 @@ List of all orders
    :query string code: Only return orders that match the given order code
    :query string status: Only return orders in the given order status (see above)
    :query string search: Only return orders matching a given search query
+   :query integer item: Only return orders with a position that contains this item ID. *Warning:* Result will also include orders if they contain mixed items, and it will even return orders where the item is only contained in a canceled position.
+   :query integer variation: Only return orders with a position that contains this variation ID. *Warning:* Result will also include orders if they contain mixed items and variations, and it will even return orders where the variation is only contained in a canceled position.
    :query boolean testmode: Only return orders with ``testmode`` set to ``true`` or ``false``
    :query boolean require_approval: If set to ``true`` or ``false``, only categories with this value for the field
                                     ``require_approval`` will be returned.

doc/api/resources/sendmail_rules.rst

@@ -0,0 +1,281 @@
+Automated email rules
+=====================
+
+Resource description
+--------------------
+
+Automated email rules that specify emails that the system will send automatically at a specific point in time, e.g.
+the day of the event.
+
+.. rst-class:: rest-resource-table
+
+===================================== ========================== =======================================================
+Field                                 Type                       Description
+===================================== ========================== =======================================================
+id                                    integer                    Internal ID of the rule
+enabled                               boolean                    If ``false``, the rule is ignored
+subject                               multi-lingual string       The subject of the email
+template                              multi-lingual string       The body of the email
+all_products                          boolean                    If ``true``, the email is sent to buyers of all products
+limit_products                        list of integers           List of product IDs, if ``all_products`` is not set
+include_pending                       boolean                    If ``true``, the email is sent to pending orders. If ``false``,
+                                                                 only paid orders are considered.
+date_is_absolute                      boolean                    If ``true``, the email is set at a specific point in time.
+send_date                             datetime                   If ``date_is_absolute`` is set: Date and time to send the email.
+send_offset_days                      integer                    If ``date_is_absolute`` is not set, this is the number of days
+                                                                 before/after the email is sent.
+send_offset_time                      time                       If ``date_is_absolute`` is not set, this is the time of day the
+                                                                 email is sent on the day specified by ``send_offset_days``.
+offset_to_event_end                   boolean                    If ``true``, ``send_offset_days`` is relative to the event end
+                                                                 date. Otherwise it is relative to the event start date.
+offset_is_after                       boolean                    If ``true``, ``send_offset_days`` is the number of days **after**
+                                                                 the event start or end date. Otherwise it is the number of days
+                                                                 **before**.
+send_to                               string                     Can be ``"orders"`` if the email should be sent to customers
+                                                                 (one email per order),
+                                                                 ``"attendees"`` if the email should be sent to every attendee,
+                                                                 or ``"both"``.
+                                                                 date. Otherwise it is relative to the event start date.
+===================================== ========================== =======================================================
+
+
+Endpoints
+---------
+
+.. http:get:: /api/v1/organizers/(organizer)/events/(event)/sendmail_rules/
+
+   Returns a list of all rules configured for an event.
+
+   **Example request**:
+
+   .. sourcecode:: http
+
+      GET /api/v1/organizers/bigevents/events/sampleconf/sendmail_rules/ HTTP/1.1
+      Host: pretix.eu
+      Accept: application/json, text/javascript
+
+   **Example response**:
+
+   .. sourcecode:: http
+
+      HTTP/1.1 200 OK
+      Vary: Accept
+      Content-Type: application/json
+
+      {
+        "count": 1,
+        "next": null,
+        "previous": null,
+        "results": [
+          {
+            "id": 1,
+            "enabled": true,
+            "subject": {"en": "See you tomorrow!"},
+            "template": {"en": "Don't forget your tickets, download them at {url}"},
+            "all_products": true,
+            "limit_products": [],
+            "include_pending": false,
+            "send_date": null,
+            "send_offset_days": 1,
+            "send_offset_time": "18:00",
+            "date_is_absolute": false,
+            "offset_to_event_end": false,
+            "offset_is_after": false,
+            "send_to": "orders"
+          }
+        ]
+      }
+
+   :query page: The page number in case of a multi-page result set, default is 1
+   :param organizer: The ``slug`` field of a valid organizer
+   :param event: The ``slug`` field of the event to fetch
+   :statuscode 200: no error
+   :statuscode 401: Authentication failure
+   :statuscode 403: The requested organizer does not exist **or** you have no permission to view it.
+
+.. http:get:: /api/v1/organizers/(organizer)/events/(event)/sendmail_rules/(id)/
+
+   Returns information on one rule, identified by its ID.
+
+   **Example request**:
+
+   .. sourcecode:: http
+
+      GET /api/v1/organizers/bigevents/events/sampleconf/sendmail_rules/1/ HTTP/1.1
+      Host: pretix.eu
+      Accept: application/json, text/javascript
+
+   **Example response**:
+
+   .. sourcecode:: http
+
+      HTTP/1.1 200 OK
+      Vary: Accept
+      Content-Type: application/json
+
+      {
+        "id": 1,
+        "enabled": true,
+        "subject": {"en": "See you tomorrow!"},
+        "template": {"en": "Don't forget your tickets, download them at {url}"},
+        "all_products": true,
+        "limit_products": [],
+        "include_pending": false,
+        "send_date": null,
+        "send_offset_days": 1,
+        "send_offset_time": "18:00",
+        "date_is_absolute": false,
+        "offset_to_event_end": false,
+        "offset_is_after": false,
+        "send_to": "orders"
+      }
+
+   :param organizer: The ``slug`` field of the organizer to fetch
+   :param event: The ``slug`` field of the event to fetch
+   :param id: The ``id`` field of the rule to fetch
+   :statuscode 200: no error
+   :statuscode 401: Authentication failure
+   :statuscode 403: The requested organizer/event/rule does not exist **or** you have no permission to view it.
+
+.. http:post:: /api/v1/organizers/(organizer)/events/(event)/sendmail_rules/
+
+   Create a new rule.
+
+   **Example request**:
+
+   .. sourcecode:: http
+
+      POST /api/v1/organizers/bigevents/events/sampleconf/sendmail_rules/ HTTP/1.1
+      Host: pretix.eu
+      Accept: application/json, text/javascript
+      Content-Type: application/json
+      Content-Length: 166
+
+      {
+        "enabled": true,
+        "subject": {"en": "See you tomorrow!"},
+        "template": {"en": "Don't forget your tickets, download them at {url}"},
+        "all_products": true,
+        "limit_products": [],
+        "include_pending": false,
+        "send_date": null,
+        "send_offset_days": 1,
+        "send_offset_time": "18:00",
+        "date_is_absolute": false,
+        "offset_to_event_end": false,
+        "offset_is_after": false,
+        "send_to": "orders"
+      }
+
+   **Example response**:
+
+   .. sourcecode:: http
+
+      HTTP/1.1 201 Created
+      Vary: Accept
+      Content-Type: application/json
+
+      {
+        "id": 1,
+        "enabled": true,
+        "subject": {"en": "See you tomorrow!"},
+        "template": {"en": "Don't forget your tickets, download them at {url}"},
+        "all_products": true,
+        "limit_products": [],
+        "include_pending": false,
+        "send_date": null,
+        "send_offset_days": 1,
+        "send_offset_time": "18:00",
+        "date_is_absolute": false,
+        "offset_to_event_end": false,
+        "offset_is_after": false,
+        "send_to": "orders"
+      }
+
+   :param organizer: The ``slug`` field of the organizer to create a rule for
+   :param event: The ``slug`` field of the event to create a rule for
+   :statuscode 201: no error
+   :statuscode 400: The rule could not be created due to invalid submitted data.
+   :statuscode 401: Authentication failure
+   :statuscode 403: The requested organizer/event does not exist **or** you have no permission to create rules.
+
+
+.. http:patch:: /api/v1/organizers/(organizer)/events/(event)/sendmail_rules/(id)/
+
+   Update a rule. You can also use ``PUT`` instead of ``PATCH``. With ``PUT``, you have to provide all fields of
+   the resource, other fields will be reset to default. With ``PATCH``, you only need to provide the fields that you
+   want to change.
+
+   **Example request**:
+
+   .. sourcecode:: http
+
+      PATCH /api/v1/organizers/bigevents/events/sampleconf/sendmail_rules/1/ HTTP/1.1
+      Host: pretix.eu
+      Accept: application/json, text/javascript
+      Content-Type: application/json
+      Content-Length: 34
+
+      {
+        "enabled": false,
+      }
+
+   **Example response**:
+
+   .. sourcecode:: http
+
+      HTTP/1.1 200 OK
+      Vary: Accept
+      Content-Type: text/javascript
+
+      {
+        "id": 1,
+        "enabled": false,
+        "subject": {"en": "See you tomorrow!"},
+        "template": {"en": "Don't forget your tickets, download them at {url}"},
+        "all_products": true,
+        "limit_products": [],
+        "include_pending": false,
+        "send_date": null,
+        "send_offset_days": 1,
+        "send_offset_time": "18:00",
+        "date_is_absolute": false,
+        "offset_to_event_end": false,
+        "offset_is_after": false,
+        "send_to": "orders"
+      }
+
+   :param organizer: The ``slug`` field of the organizer to modify
+   :param event: The ``slug`` field of the event to modify
+   :param id: The ``id`` field of the rule to modify
+   :statuscode 200: no error
+   :statuscode 400: The rule could not be modified due to invalid submitted data.
+   :statuscode 401: Authentication failure
+   :statuscode 403: The requested organizer/event/rule does not exist **or** you have no permission to change it.
+
+
+.. http:delete:: /api/v1/organizers/(organizer)/events/(event)/sendmail_rules/(id)/
+
+   Delete a rule.
+
+   **Example request**:
+
+   .. sourcecode:: http
+
+      DELETE /api/v1/organizers/bigevents/events/sampleconf/sendmail_rules/1/ HTTP/1.1
+      Host: pretix.eu
+      Accept: application/json, text/javascript
+
+   **Example response**:
+
+   .. sourcecode:: http
+
+      HTTP/1.1 204 No Content
+      Vary: Accept
+
+   :param organizer: The ``slug`` field of the organizer to modify
+   :param event: The ``slug`` field of the event to modify
+   :param id: The ``id`` field of the rule to delete
+   :statuscode 204: no error
+   :statuscode 401: Authentication failure
+   :statuscode 403: The requested organizer/event/rule does not exist **or** you have no permission to change it **or** this rule cannot be deleted since it is currently in use.

doc/development/contribution/style.rst

@@ -1,6 +1,11 @@
+.. spelling:: Rebase rebasing
+
 Coding style and quality
 ========================
 
+Code
+----
+
 * Basically, we want all python code to follow the `PEP 8`_ standard. There are a few exceptions where
   we see things differently or just aren't that strict. The ``setup.cfg`` file in the project's source
   folder contains definitions that allow `flake8`_ to check for violations automatically. See :ref:`checksandtests`
@@ -20,8 +25,62 @@ Coding style and quality
   test suite are in the style of Python's unit test module. If you extend those files, you might continue in this style,
   but please use ``pytest`` style for any new test files.
 
-* Please keep the first line of your commit messages short. When referencing an issue, please phrase it like
-  ``Fix #123 -- Problems with order creation`` or ``Refs #123 -- Fix this part of that bug``.
+Commits and Pull Requests
+-------------------------
+
+
+
+Most commits should start as pull requests, therefore this applies to the titles of pull requests as well since
+the pull request title will become the commit message on merge. We prefer merging with GitHub's "Squash and merge"
+feature if the PR contains multiple commits that do not carry value to keep. If there is value in keeping the
+individual commits, we use "Rebase and merge" instead. Merge commits should be avoided.
+
+* The commit message should start with a single subject line and can optionally be followed by a commit message body.
+
+  * The subject line should be the shortest possible representation of what the commit changes. Someone who reviewed
+    the commit should able to immediately remember the commit in a couple of weeks based on the subject line and tell
+    it apart from other commits.
+
+  * If there's additional useful information that we should keep, such as reasoning behind the commit, you can
+    add a longer body, separated from the first line by a blank line.
+
+  * The body should explain **what** you changed and more importantly **why** you changed it. There's no need to iterate
+    **how** you changed something.
+
+* The subject line should be capitalized ("Add new feature" instead of "add new feature") and should not end with a period
+  ("Add new feature" instead of "Add new feature.")
+
+* The subject line should be written in imperative mood, as if you were giving a command what the computer should do if the
+  commit is applied. This is how generated commit messages by git itself are already written ("Merge branch …", "Revert …")
+  and makes for short and consistent messages.
+
+  * Good: "Fix typo in template"
+  * Good: "Add Chinese translation"
+  * Good: "Remove deprecated method"
+  * Good: "Bump version to 4.4.0"
+  * Bad: "Fixed bug with …"
+  * Bad: "Fixes bug with …"
+  * Bad: "Fixing bug …"
+
+* If all changes in your commit are in context of a single feature or e.g. a bundled plugin, it makes sense to prefix the
+  subject line with the name of that feature. Examples:
+
+  * "API: Add support for PATCH on customers"
+  * "Docs: Add chapter on alpaca feeding"
+  * "Stripe: Fix duplicate payments"
+  * "Order change form: Fix incorrect validation"
+
+* If your commit references a GitHub issue that is fully resolved by your commit, start your subject line with the issue
+  ID in the form of "Fix #1234 -- Crash in order list". In this case, you can omit the verb "Fix" at the beginning of the
+  second part of the message to avoid repetition of the word "fix". If your commit only partially resolves the issue, use
+  "Refs #1234 -- Crash in order list" instead.
+
+* Applies to pretix employees only: If your commit references a sentry issue, please put it in parentheses at the end
+  of the subject line or inside the body ("Fix crash in order list (PRETIXEU-ABC)"). If your commit references a support
+  ticket, please put it in parentheses at the end of the subject line with a "Z#" prefix ("Fix crash in order list (Z#12345)").
+
+* If your PR was open for a while and might cause conflicts on merge, please prefer rebasing it (``git rebase -i master``)
+  over merging ``master`` into your branch unless it is prohibitively complicated.
 
 
 .. _PEP 8: https://legacy.python.org/dev/peps/pep-0008/

doc/development/implementation/models.rst

@@ -92,6 +92,9 @@ Carts and Orders
 .. autoclass:: pretix.base.models.OrderRefund
    :members:
 
+.. autoclass:: pretix.base.models.Transaction
+   :members:
+
 .. autoclass:: pretix.base.models.CartPosition
    :members:
 

doc/development/setup.rst

@@ -26,7 +26,7 @@ Your should install the following on your system:
 * ``libssl`` (Debian package: ``libssl-dev``)
 * ``libxml2`` (Debian package ``libxml2-dev``)
 * ``libxslt`` (Debian package ``libxslt1-dev``)
-* ``libenchant1c2a`` (Debian package ``libenchant1c2a``)
+* ``libenchant-2-2`` (Debian package ``libenchant-2-2``)
 * ``msgfmt`` (Debian package ``gettext``)
 * ``git``
 
@@ -51,7 +51,12 @@ the dependencies might fail::
 
 Working with the code
 ---------------------
-The first thing you need are all the main application's dependencies::
+If you do not have a recent installation of ``nodejs``, install it now::
+
+    curl -sL https://deb.nodesource.com/setup_17.x | sudo -E bash -
+    sudo apt install nodejs
+
+To make sure it is on your path variable, close and reopen your terminal. Now, install the Python-level dependencies of pretix::
 
     cd src/
     pip3 install -e ".[dev]"

doc/spelling_wordlist.txt

@@ -17,6 +17,7 @@ bic
 BIC
 boolean
 booleans
+bugfix
 cancelled
 casted
 Ceph
@@ -77,6 +78,7 @@ mixin
 mixins
 multi
 multidomain
+multiplicator
 namespace
 namespaced
 namespaces

src/.update-locales

@@ -34,5 +34,7 @@ git push
 # Unlock Weblate
 for c in $COMPONENTS; do
 	wlc unlock $c;
+done
+for c in $COMPONENTS; do
 	wlc pull $c;
 done

src/pretix/__init__.py

@@ -19,4 +19,4 @@
 # You should have received a copy of the GNU Affero General Public License along with this program.  If not, see
 # <https://www.gnu.org/licenses/>.
 #
-__version__ = "4.3.0"
+__version__ = "4.5.0.dev0"

src/pretix/api/serializers/event.py

@@ -704,6 +704,7 @@ class EventSettingsSerializer(SettingsSerializer):
         'payment_term_accept_late',
         'payment_explanation',
         'payment_pending_hidden',
+        'mail_days_order_expire_warning',
         'ticket_download',
         'ticket_download_date',
         'ticket_download_addons',
@@ -733,6 +734,7 @@ class EventSettingsSerializer(SettingsSerializer):
         'invoice_numbers_prefix_cancellations',
         'invoice_numbers_counter_length',
         'invoice_attendee_name',
+        'invoice_event_location',
         'invoice_include_expire_date',
         'invoice_address_explanation_text',
         'invoice_email_attachment',
@@ -762,6 +764,7 @@ class EventSettingsSerializer(SettingsSerializer):
         'cancel_allow_user_paid_refund_as_giftcard',
         'cancel_allow_user_paid_require_approval',
         'change_allow_user_variation',
+        'change_allow_user_addons',
         'change_allow_user_until',
         'change_allow_user_price',
         'primary_color',

src/pretix/api/serializers/item.py

@@ -59,7 +59,7 @@ class InlineItemVariationSerializer(I18nAwareModelSerializer):
         model = ItemVariation
         fields = ('id', 'value', 'active', 'description',
                   'position', 'default_price', 'price', 'original_price',
-                  'require_membership', 'require_membership_types', 'available_from', 'available_until',
+                  'require_membership', 'require_membership_types', 'require_membership_hidden', 'available_from', 'available_until',
                   'sales_channels', 'hide_without_voucher',)
 
     def __init__(self, *args, **kwargs):
@@ -75,7 +75,7 @@ class ItemVariationSerializer(I18nAwareModelSerializer):
         model = ItemVariation
         fields = ('id', 'value', 'active', 'description',
                   'position', 'default_price', 'price', 'original_price',
-                  'require_membership', 'require_membership_types', 'available_from', 'available_until',
+                  'require_membership', 'require_membership_types', 'require_membership_hidden', 'available_from', 'available_until',
                   'sales_channels', 'hide_without_voucher',)
 
     def __init__(self, *args, **kwargs):
@@ -175,7 +175,7 @@ class ItemSerializer(I18nAwareModelSerializer):
                   'min_per_order', 'max_per_order', 'checkin_attention', 'has_variations', 'variations',
                   'addons', 'bundles', 'original_price', 'require_approval', 'generate_tickets',
                   'show_quota_left', 'hidden_if_available', 'allow_waitinglist', 'issue_giftcard', 'meta_data',
-                  'require_membership', 'require_membership_types', 'grant_membership_type',
+                  'require_membership', 'require_membership_types', 'require_membership_hidden', 'grant_membership_type',
                   'grant_membership_duration_like_event', 'grant_membership_duration_days',
                   'grant_membership_duration_months')
         read_only_fields = ('has_variations',)

src/pretix/api/serializers/order.py

@@ -1404,6 +1404,7 @@ class OrderCreateSerializer(I18nAwareModelSerializer):
                 state=OrderPayment.PAYMENT_STATE_CREATED
             )
 
+        order.create_transactions(is_new=True, fees=fees, positions=pos_map.values())
         return order
 
 
@@ -1427,7 +1428,7 @@ class InlineInvoiceLineSerializer(I18nAwareModelSerializer):
         model = InvoiceLine
         fields = ('position', 'description', 'item', 'variation', 'attendee_name', 'event_date_from',
                   'event_date_to', 'gross_value', 'tax_value', 'tax_rate', 'tax_name', 'fee_type',
-                  'fee_internal_type')
+                  'fee_internal_type', 'event_location')
 
 
 class InvoiceSerializer(I18nAwareModelSerializer):

src/pretix/api/serializers/organizer.py

@@ -296,7 +296,14 @@ class OrganizerSettingsSerializer(SettingsSerializer):
         'theme_round_borders',
         'primary_font',
         'organizer_logo_image_inherit',
-        'organizer_logo_image'
+        'organizer_logo_image',
+        'privacy_url',
+        'cookie_consent',
+        'cookie_consent_dialog_title',
+        'cookie_consent_dialog_text',
+        'cookie_consent_dialog_text_secondary',
+        'cookie_consent_dialog_button_yes',
+        'cookie_consent_dialog_button_no',
     ]
 
     def __init__(self, *args, **kwargs):

src/pretix/api/views/checkin.py

@@ -430,7 +430,13 @@ class CheckinListPositionViewSet(viewsets.ReadOnlyModelViewSet):
             if self.kwargs['pk'].isnumeric():
                 op = queryset.get(Q(pk=self.kwargs['pk']) | Q(secret=self.kwargs['pk']))
             else:
-                op = queryset.get(secret=self.kwargs['pk'])
+                # In application/x-www-form-urlencoded, you can encodes space ' ' with '+' instead of '%20'.
+                # `id`, however, is part of a path where this technically is not allowed. Old versions of our
+                # scan apps still do it, so we try work around it!
+                try:
+                    op = queryset.get(secret=self.kwargs['pk'])
+                except OrderPosition.DoesNotExist:
+                    op = queryset.get(secret=self.kwargs['pk'].replace('+', ' '))
         except OrderPosition.DoesNotExist:
             revoked_matches = list(self.request.event.revoked_secrets.filter(secret=self.kwargs['pk']))
             if len(revoked_matches) == 0:

src/pretix/api/views/exporters.py

@@ -69,7 +69,7 @@ class ExportersMixin:
         cf = get_object_or_404(CachedFile, id=kwargs['cfid'])
         if cf.file:
             resp = ChunkBasedFileResponse(cf.file.file, content_type=cf.type)
-            resp['Content-Disposition'] = 'attachment; filename="{}"'.format(cf.filename)
+            resp['Content-Disposition'] = 'attachment; filename="{}"'.format(cf.filename).encode("ascii", "ignore")
             return resp
         elif not settings.HAS_CELERY:
             return Response(

src/pretix/api/views/order.py

@@ -92,6 +92,8 @@ with scopes_disabled():
         subevent_after = django_filters.IsoDateTimeFilter(method='subevent_after_qs')
         subevent_before = django_filters.IsoDateTimeFilter(method='subevent_before_qs')
         search = django_filters.CharFilter(method='search_qs')
+        item = django_filters.CharFilter(field_name='all_positions', lookup_expr='item_id')
+        variation = django_filters.CharFilter(field_name='all_positions', lookup_expr='variation_id')
 
         class Meta:
             model = Order
@@ -214,7 +216,9 @@ class OrderViewSet(viewsets.ModelViewSet):
                     'positions',
                     opq.all().prefetch_related(
                         Prefetch('checkins', queryset=Checkin.objects.all()),
-                        'item', 'variation', 'answers', 'answers__options', 'answers__question', 'seat',
+                        'item', 'variation',
+                        Prefetch('answers', queryset=QuestionAnswer.objects.prefetch_related('options', 'question').order_by('question__position')),
+                        'seat',
                     )
                 )
             )

src/pretix/base/apps.py

@@ -47,6 +47,7 @@ class PretixBaseConfig(AppConfig):
         from . import notifications  # NOQA
         from . import email  # NOQA
         from .services import auth, checkin, export, mail, tickets, cart, orderimport, orders, invoices, cleanup, update_check, quotas, notifications, vouchers  # NOQA
+        from .models import _transactions  # NOQA
         from django.conf import settings
 
         try:

src/pretix/base/email.py

@@ -539,6 +539,22 @@ def base_placeholders(sender, **kwargs):
             'voucher_list', ['voucher_list'], lambda voucher_list: '  \n'.join(voucher_list),
             '    68CYU2H6ZTP3WLK5\n    7MB94KKPVEPSMVF2'
         ),
+        SimpleFunctionalMailTextPlaceholder(
+            # join vouchers with two spaces at end of line so markdown-parser inserts a <br>
+            'voucher_url_list', ['event', 'voucher_list'],
+            lambda event, voucher_list: '  \n'.join([
+                build_absolute_uri(
+                    event, 'presale:event.redeem'
+                ) + '?voucher=' + c
+                for c in voucher_list
+            ]),
+            lambda event: '  \n'.join([
+                build_absolute_uri(
+                    event, 'presale:event.redeem'
+                ) + '?voucher=' + c
+                for c in ['68CYU2H6ZTP3WLK5', '7MB94KKPVEPSMVF2']
+            ]),
+        ),
         SimpleFunctionalMailTextPlaceholder(
             'url', ['event', 'voucher_list'], lambda event, voucher_list: build_absolute_uri(event, 'presale:event.index', kwargs={
                 'event': event.slug,

src/pretix/base/exporters/invoices.py

@@ -324,7 +324,6 @@ class InvoiceDataExporter(InvoiceExporterMixin, MultiSheetListExporter):
                 _('Tax rate'),
                 _('Tax name'),
                 _('Event start date'),
-
                 _('Date'),
                 _('Order code'),
                 _('E-mail address'),
@@ -348,6 +347,8 @@ class InvoiceDataExporter(InvoiceExporterMixin, MultiSheetListExporter):
                 _('Invoice recipient:') + ' ' + _('Beneficiary'),
                 _('Invoice recipient:') + ' ' + _('Internal reference'),
                 _('Payment providers'),
+                _('Event end date'),
+                _('Location'),
             ]
 
             p_providers = OrderPayment.objects.filter(
@@ -406,7 +407,9 @@ class InvoiceDataExporter(InvoiceExporterMixin, MultiSheetListExporter):
                     ', '.join([
                         str(self.providers.get(p, p)) for p in sorted(set((l.payment_providers or '').split(',')))
                         if p and p != 'free'
-                    ])
+                    ]),
+                    date_format(l.event_date_to, "SHORT_DATE_FORMAT") if l.event_date_to else "",
+                    l.event_location or "",
                 ]
 
     @cached_property

src/pretix/base/exporters/json.py

@@ -55,16 +55,20 @@ class JSONExporter(BaseExporter):
                     'name': str(self.event.organizer.name),
                     'slug': self.event.organizer.slug
                 },
+                'meta_data': self.event.meta_data,
                 'categories': [
                     {
                         'id': category.id,
                         'name': str(category.name),
+                        'description': str(category.description),
+                        'position': category.position,
                         'internal_name': category.internal_name
                     } for category in self.event.categories.all()
                 ],
                 'items': [
                     {
                         'id': item.id,
+                        'position': item.position,
                         'name': str(item.name),
                         'internal_name': str(item.internal_name),
                         'category': item.category_id,
@@ -73,13 +77,35 @@ class JSONExporter(BaseExporter):
                         'tax_name': str(item.tax_rule.name) if item.tax_rule else None,
                         'admission': item.admission,
                         'active': item.active,
+                        'sales_channels': item.sales_channels,
+                        'description': str(item.description),
+                        'available_from': item.available_from,
+                        'available_until': item.available_until,
+                        'require_voucher': item.require_voucher,
+                        'hide_without_voucher': item.hide_without_voucher,
+                        'allow_cancel': item.allow_cancel,
+                        'require_bundling': item.require_bundling,
+                        'min_per_order': item.min_per_order,
+                        'max_per_order': item.max_per_order,
+                        'checkin_attention': item.checkin_attention,
+                        'original_price': item.original_price,
+                        'issue_giftcard': item.issue_giftcard,
+                        'meta_data': item.meta_data,
+                        'require_membership': item.require_membership,
                         'variations': [
                             {
                                 'id': variation.id,
                                 'active': variation.active,
                                 'price': variation.default_price if variation.default_price is not None else
                                 item.default_price,
-                                'name': str(variation)
+                                'name': str(variation),
+                                'description': str(variation.description),
+                                'position': variation.position,
+                                'require_membership': variation.require_membership,
+                                'sales_channels': variation.sales_channels,
+                                'available_from': variation.available_from,
+                                'available_until': variation.available_until,
+                                'hide_without_voucher': variation.hide_without_voucher,
                             } for variation in item.variations.all()
                         ]
                     } for item in self.event.items.select_related('tax_rule').prefetch_related('variations')
@@ -87,7 +113,13 @@ class JSONExporter(BaseExporter):
                 'questions': [
                     {
                         'id': question.id,
+                        'identifier': question.identifier,
+                        'required': question.required,
                         'question': str(question.question),
+                        'position': question.position,
+                        'hidden': question.hidden,
+                        'ask_during_checkin': question.ask_during_checkin,
+                        'help_text': str(question.help_text),
                         'type': question.type
                     } for question in self.event.questions.all()
                 ],
@@ -95,7 +127,18 @@ class JSONExporter(BaseExporter):
                     {
                         'code': order.code,
                         'status': order.status,
+                        'customer': order.customer.identifier if order.customer else None,
+                        'testmode': order.testmode,
                         'user': order.email,
+                        'email': order.email,
+                        'phone': str(order.phone),
+                        'locale': order.locale,
+                        'comment': order.comment,
+                        'custom_followup_at': order.custom_followup_at,
+                        'require_approval': order.require_approval,
+                        'checkin_attention': order.checkin_attention,
+                        'sales_channel': order.sales_channel,
+                        'expires': order.expires,
                         'datetime': order.datetime,
                         'fees': [
                             {
@@ -108,11 +151,21 @@ class JSONExporter(BaseExporter):
                         'positions': [
                             {
                                 'id': position.id,
+                                'positionid': position.positionid,
                                 'item': position.item_id,
                                 'variation': position.variation_id,
+                                'subevent': position.subevent_id,
+                                'seat': position.seat.seat_guid if position.seat else None,
                                 'price': position.price,
+                                'tax_rate': position.tax_rate,
+                                'tax_value': position.tax_value,
                                 'attendee_name': position.attendee_name,
                                 'attendee_email': position.attendee_email,
+                                'company': position.company,
+                                'street': position.street,
+                                'zipcode': position.zipcode,
+                                'country': str(position.country) if position.country else None,
+                                'state': position.state,
                                 'secret': position.secret,
                                 'addon_to': position.addon_to_id,
                                 'answers': [
@@ -124,15 +177,30 @@ class JSONExporter(BaseExporter):
                             } for position in order.positions.all()
                         ]
                     } for order in
-                    self.event.orders.all().prefetch_related('positions', 'positions__answers', 'fees')
+                    self.event.orders.all().prefetch_related('positions', 'positions__answers', 'positions__seat', 'customer', 'fees')
                 ],
                 'quotas': [
                     {
                         'id': quota.id,
                         'size': quota.size,
+                        'subevent': quota.subevent_id,
                         'items': [item.id for item in quota.items.all()],
                         'variations': [variation.id for variation in quota.variations.all()],
                     } for quota in self.event.quotas.all().prefetch_related('items', 'variations')
+                ],
+                'subevents': [
+                    {
+                        'id': se.id,
+                        'name': str(se.name),
+                        'location': str(se.location),
+                        'date_from': se.date_from,
+                        'date_to': se.date_to,
+                        'date_admission': se.date_admission,
+                        'geo_lat': se.geo_lat,
+                        'geo_lon': se.geo_lon,
+                        'is_public': se.is_public,
+                        'meta_data': se.meta_data,
+                    } for se in self.event.subevents.all()
                 ]
             }
         }

src/pretix/base/exporters/orderlist.py

@@ -33,6 +33,7 @@
 # License for the specific language governing permissions and limitations under the License.
 
 from collections import OrderedDict
+from datetime import date, datetime, time
 from decimal import Decimal
 
 import dateutil
@@ -42,10 +43,10 @@ from django.db.models import (
     Case, CharField, Count, DateTimeField, F, IntegerField, Max, Min, OuterRef,
     Q, Subquery, Sum, When,
 )
-from django.db.models.functions import Coalesce, TruncDate
+from django.db.models.functions import Coalesce
 from django.dispatch import receiver
 from django.utils.functional import cached_property
-from django.utils.timezone import get_current_timezone, now
+from django.utils.timezone import get_current_timezone, make_aware, now
 from django.utils.translation import gettext as _, gettext_lazy, pgettext
 
 from pretix.base.models import (
@@ -181,41 +182,43 @@ class OrderListExporter(MultiSheetListExporter):
 
         if form_data.get('date_from'):
             date_value = form_data.get('date_from')
-            if isinstance(date_value, str):
+            if not isinstance(date_value, date):
                 date_value = dateutil.parser.parse(date_value).date()
+            datetime_value = make_aware(datetime.combine(date_value, time(0, 0, 0)), self.timezone)
 
-            annotations['date'] = TruncDate(f'{rel}datetime')
-            filters['date__gte'] = date_value
+            filters[f'{rel}datetime__gte'] = datetime_value
 
         if form_data.get('date_to'):
             date_value = form_data.get('date_to')
-            if isinstance(date_value, str):
+            if not isinstance(date_value, date):
                 date_value = dateutil.parser.parse(date_value).date()
+            datetime_value = make_aware(datetime.combine(date_value, time(23, 59, 59, 999999)), self.timezone)
 
-            annotations['date'] = TruncDate(f'{rel}datetime')
-            filters['date__lte'] = date_value
+            filters[f'{rel}datetime__lte'] = datetime_value
 
         if form_data.get('event_date_from'):
             date_value = form_data.get('event_date_from')
-            if isinstance(date_value, str):
+            if not isinstance(date_value, date):
                 date_value = dateutil.parser.parse(date_value).date()
+            datetime_value = make_aware(datetime.combine(date_value, time(0, 0, 0)), self.timezone)
 
             annotations['event_date_max'] = Case(
                 When(**{f'{rel}event__has_subevents': True}, then=Max(f'{rel}all_positions__subevent__date_from')),
                 default=F(f'{rel}event__date_from'),
             )
-            filters['event_date_max__gte'] = date_value
+            filters['event_date_max__gte'] = datetime_value
 
         if form_data.get('event_date_to'):
             date_value = form_data.get('event_date_to')
-            if isinstance(date_value, str):
+            if not isinstance(date_value, date):
                 date_value = dateutil.parser.parse(date_value).date()
+            datetime_value = make_aware(datetime.combine(date_value, time(23, 59, 59, 999999)), self.timezone)
 
             annotations['event_date_min'] = Case(
                 When(**{f'{rel}event__has_subevents': True}, then=Min(f'{rel}all_positions__subevent__date_from')),
                 default=F(f'{rel}event__date_from'),
             )
-            filters['event_date_min__lte'] = date_value
+            filters['event_date_min__lte'] = datetime_value
 
         if filters:
             return qs.annotate(**annotations).filter(**filters)
@@ -870,6 +873,78 @@ class QuotaListExporter(ListExporter):
         return '{}_quotas'.format(self.event.slug)
 
 
+def generate_GiftCardTransactionListExporter(organizer):  # hackhack
+    class GiftcardTransactionListExporter(ListExporter):
+        identifier = 'giftcardtransactionlist'
+        verbose_name = gettext_lazy('Gift card transactions')
+
+        @property
+        def additional_form_fields(self):
+            d = [
+                ('date_from',
+                 forms.DateField(
+                     label=_('Start date'),
+                     widget=forms.DateInput(attrs={'class': 'datepickerfield'}),
+                     required=False,
+                 )),
+                ('date_to',
+                 forms.DateField(
+                     label=_('End date'),
+                     widget=forms.DateInput(attrs={'class': 'datepickerfield'}),
+                     required=False,
+                 )),
+            ]
+            d = OrderedDict(d)
+            return d
+
+        def iterate_list(self, form_data):
+            qs = GiftCardTransaction.objects.filter(
+                card__issuer=organizer,
+            ).order_by('datetime').select_related('card', 'order', 'order__event')
+
+            if form_data.get('date_from'):
+                date_value = form_data.get('date_from')
+                if isinstance(date_value, str):
+                    date_value = dateutil.parser.parse(date_value).date()
+                qs = qs.filter(
+                    datetime__gte=make_aware(datetime.combine(date_value, time(0, 0, 0)), self.timezone)
+                )
+
+            if form_data.get('date_to'):
+                date_value = form_data.get('date_to')
+                if isinstance(date_value, str):
+                    date_value = dateutil.parser.parse(date_value).date()
+
+                qs = qs.filter(
+                    datetime__lte=make_aware(datetime.combine(date_value, time(23, 59, 59, 999999)), self.timezone)
+                )
+
+            headers = [
+                _('Gift card code'),
+                _('Test mode'),
+                _('Date'),
+                _('Amount'),
+                _('Currency'),
+                _('Order'),
+            ]
+            yield headers
+
+            for obj in qs:
+                row = [
+                    obj.card.secret,
+                    _('TEST MODE') if obj.card.testmode else '',
+                    obj.datetime.astimezone(self.timezone).strftime('%Y-%m-%d %H:%M:%S'),
+                    obj.value,
+                    obj.card.currency,
+                    obj.order.full_code if obj.order else None,
+                ]
+                yield row
+
+        def get_filename(self):
+            return '{}_giftcardtransactions'.format(organizer.slug)
+    return GiftcardTransactionListExporter
+
+
 class GiftcardRedemptionListExporter(ListExporter):
     identifier = 'giftcardredemptionlist'
     verbose_name = gettext_lazy('Gift card redemptions')
@@ -1062,3 +1137,8 @@ def register_multievent_i_giftcardredemptionlist_exporter(sender, **kwargs):
 @receiver(register_multievent_data_exporters, dispatch_uid="multiexporter_giftcardlist")
 def register_multievent_i_giftcardlist_exporter(sender, **kwargs):
     return generate_GiftCardListExporter(sender)
+
+
+@receiver(register_multievent_data_exporters, dispatch_uid="multiexporter_giftcardtransactionlist")
+def register_multievent_i_giftcardtransactionlist_exporter(sender, **kwargs):
+    return generate_GiftCardTransactionListExporter(sender)

src/pretix/base/forms/questions.py

@@ -37,13 +37,10 @@ import json
 import logging
 from decimal import Decimal
 from io import BytesIO
-from urllib.error import HTTPError
 
 import dateutil.parser
 import pycountry
 import pytz
-import vat_moss.errors
-import vat_moss.id
 from babel import Locale
 from django import forms
 from django.conf import settings
@@ -52,7 +49,7 @@ from django.core.exceptions import ValidationError
 from django.core.files.uploadedfile import SimpleUploadedFile
 from django.core.validators import MaxValueValidator, MinValueValidator
 from django.db.models import QuerySet
-from django.forms import Select
+from django.forms import Select, widgets
 from django.utils import translation
 from django.utils.formats import date_format
 from django.utils.html import escape
@@ -76,8 +73,9 @@ from pretix.base.i18n import (
     get_babel_locale, get_language_without_region, language,
 )
 from pretix.base.models import InvoiceAddress, Question, QuestionOption
-from pretix.base.models.tax import (
-    EU_COUNTRIES, cc_to_vat_prefix, is_eu_country,
+from pretix.base.models.tax import VAT_ID_COUNTRIES, ask_for_vat_id
+from pretix.base.services.tax import (
+    VATIDFinalError, VATIDTemporaryError, validate_vat_id,
 )
 from pretix.base.settings import (
     COUNTRIES_WITH_STATE_IN_ADDRESS, PERSON_NAME_SALUTATIONS,
@@ -154,8 +152,9 @@ class NamePartsWidget(forms.MultiWidget):
                     final_attrs,
                     id='%s_%s' % (id_, i),
                     title=self.scheme['fields'][i][1],
-                    placeholder=self.scheme['fields'][i][1],
                 )
+                if not isinstance(widget, widgets.Select):
+                    these_attrs['placeholder'] = self.scheme['fields'][i][1]
                 if self.scheme['fields'][i][0] in REQUIRED_NAME_PARTS:
                     if self.field.required:
                         these_attrs['required'] = 'required'
@@ -901,7 +900,7 @@ class BaseInvoiceAddressForm(forms.ModelForm):
                 'data-display-dependency': '#id_is_business_1',
                 'autocomplete': 'organization',
             }),
-            'vat_id': forms.TextInput(attrs={'data-display-dependency': '#id_is_business_1', 'data-countries-in-eu': ','.join(EU_COUNTRIES)}),
+            'vat_id': forms.TextInput(attrs={'data-display-dependency': '#id_is_business_1', 'data-countries-with-vat-id': ','.join(VAT_ID_COUNTRIES)}),
             'internal_reference': forms.TextInput,
         }
         labels = {
@@ -921,6 +920,18 @@ class BaseInvoiceAddressForm(forms.ModelForm):
         super().__init__(*args, **kwargs)
         if not event.settings.invoice_address_vatid:
             del self.fields['vat_id']
+        elif self.validate_vat_id:
+            self.fields['vat_id'].help_text = '<br/>'.join([
+                str(_('Optional, but depending on the country you reside in we might need to charge you '
+                      'additional taxes if you do not enter it.')),
+                str(_('If you are registered in Switzerland, you can enter your UID instead.')),
+            ])
+        else:
+            self.fields['vat_id'].help_text = '<br/>'.join([
+                str(_('Optional, but it might be required for you to claim tax benefits on your invoice '
+                      'depending on your and the seller’s country of residence.')),
+                str(_('If you are registered in Switzerland, you can enter your UID instead.')),
+            ])
 
         self.fields['country'].choices = CachedCountries()
 
@@ -952,7 +963,7 @@ class BaseInvoiceAddressForm(forms.ModelForm):
         self.fields['state'].widget.is_required = True
 
         # Without JavaScript the VAT ID field is not hidden, so we empty the field if a country outside the EU is selected.
-        if cc and not is_eu_country(cc) and fprefix + 'vat_id' in self.data:
+        if cc and not ask_for_vat_id(cc) and fprefix + 'vat_id' in self.data:
             self.data = self.data.copy()
             del self.data[fprefix + 'vat_id']
 
@@ -1002,7 +1013,7 @@ class BaseInvoiceAddressForm(forms.ModelForm):
         if not data.get('is_business'):
             data['company'] = ''
             data['vat_id'] = ''
-        if data.get('is_business') and not is_eu_country(data.get('country')):
+        if data.get('is_business') and not ask_for_vat_id(data.get('country')):
             data['vat_id'] = ''
         if self.event.settings.invoice_address_required:
             if data.get('is_business') and not data.get('company'):
@@ -1025,36 +1036,19 @@ class BaseInvoiceAddressForm(forms.ModelForm):
             # Do not save the country if it is the only field set -- we don't know the user even checked it!
             self.cleaned_data['country'] = ''
 
-        if data.get('vat_id') and is_eu_country(data.get('country')) and data.get('vat_id')[:2] != cc_to_vat_prefix(str(data.get('country'))):
-            raise ValidationError(_('Your VAT ID does not match the selected country.'))
-
         if self.validate_vat_id and self.instance.vat_id_validated and 'vat_id' not in self.changed_data:
             pass
-        elif self.validate_vat_id and data.get('is_business') and is_eu_country(data.get('country')) and data.get('vat_id'):
+        elif self.validate_vat_id and data.get('is_business') and ask_for_vat_id(data.get('country')) and data.get('vat_id'):
             try:
-                result = vat_moss.id.validate(data.get('vat_id'))
-                if result:
-                    country_code, normalized_id, company_name = result
-                    self.instance.vat_id_validated = True
-                    self.instance.vat_id = normalized_id
-            except (vat_moss.errors.InvalidError, ValueError):
-                raise ValidationError(_('This VAT ID is not valid. Please re-check your input.'))
-            except vat_moss.errors.WebServiceUnavailableError:
-                logger.exception('VAT ID checking failed for country {}'.format(data.get('country')))
+                normalized_id = validate_vat_id(data.get('vat_id'), str(data.get('country')))
+                self.instance.vat_id_validated = True
+                self.instance.vat_id = normalized_id
+            except VATIDFinalError as e:
+                raise ValidationError(e.message)
+            except VATIDTemporaryError as e:
                 self.instance.vat_id_validated = False
                 if self.request and self.vat_warning:
-                    messages.warning(self.request, _('Your VAT ID could not be checked, as the VAT checking service of '
-                                                     'your country is currently not available. We will therefore '
-                                                     'need to charge VAT on your invoice. You can get the tax amount '
-                                                     'back via the VAT reimbursement process.'))
-            except (vat_moss.errors.WebServiceError, HTTPError):
-                logger.exception('VAT ID checking failed for country {}'.format(data.get('country')))
-                self.instance.vat_id_validated = False
-                if self.request and self.vat_warning:
-                    messages.warning(self.request, _('Your VAT ID could not be checked, as the VAT checking service of '
-                                                     'your country returned an incorrect result. We will therefore '
-                                                     'need to charge VAT on your invoice. Please contact support to '
-                                                     'resolve this manually.'))
+                    messages.warning(self.request, e.message)
         else:
             self.instance.vat_id_validated = False
 

src/pretix/base/forms/user.py

@@ -55,6 +55,7 @@ class UserSettingsForm(forms.ModelForm):
         'pw_current_wrong': _("The current password you entered was not correct."),
         'pw_mismatch': _("Please enter the same password twice"),
         'rate_limit': _("For security reasons, please wait 5 minutes before you try again."),
+        'pw_equal': _("Please choose a password different to your current one.")
     }
 
     old_pw = forms.CharField(max_length=255,
@@ -158,6 +159,12 @@ class UserSettingsForm(forms.ModelForm):
                 code='pw_current'
             )
 
+        if password1 and password1 == old_pw:
+            raise forms.ValidationError(
+                self.error_messages['pw_equal'],
+                code='pw_equal'
+            )
+
         if password1:
             self.instance.set_password(password1)
 

src/pretix/base/invoice.py

@@ -395,7 +395,13 @@ class ClassicInvoiceRenderer(BaseReportlabInvoiceRenderer):
             return txt
 
         if not self.invoice.event.has_subevents and self.invoice.event.settings.show_dates_on_frontpage:
-            if self.invoice.event.settings.show_date_to and self.invoice.event.date_to:
+            tz = self.invoice.event.timezone
+            show_end_date = (
+                self.invoice.event.settings.show_date_to and
+                self.invoice.event.date_to and
+                self.invoice.event.date_to.astimezone(tz).date() != self.invoice.event.date_from.astimezone(tz).date()
+            )
+            if show_end_date:
                 p_str = (
                     shorten(self.invoice.event.name) + '\n' +
                     pgettext('invoice', '{from_date}\nuntil {to_date}').format(
@@ -550,7 +556,10 @@ class ClassicInvoiceRenderer(BaseReportlabInvoiceRenderer):
         for line in self.invoice.lines.all():
             if has_taxes:
                 tdata.append((
-                    Paragraph(line.description, self.stylesheet['Normal']),
+                    Paragraph(
+                        bleach.clean(line.description, tags=['br']).strip().replace('<br>', '<br/>').replace('\n', '<br />\n'),
+                        self.stylesheet['Normal']
+                    ),
                     "1",
                     localize(line.tax_rate) + " %",
                     money_filter(line.net_value, self.invoice.event.currency),
@@ -558,7 +567,10 @@ class ClassicInvoiceRenderer(BaseReportlabInvoiceRenderer):
                 ))
             else:
                 tdata.append((
-                    Paragraph(line.description, self.stylesheet['Normal']),
+                    Paragraph(
+                        bleach.clean(line.description, tags=['br']).strip().replace('<br>', '<br/>').replace('\n', '<br />\n'),
+                        self.stylesheet['Normal']
+                    ),
                     "1",
                     money_filter(line.gross_value, self.invoice.event.currency),
                 ))
@@ -595,7 +607,7 @@ class ClassicInvoiceRenderer(BaseReportlabInvoiceRenderer):
         table.setStyle(TableStyle(tstyledata))
         story.append(table)
 
-        story.append(Spacer(1, 15 * mm))
+        story.append(Spacer(1, 10 * mm))
 
         if self.invoice.payment_provider_text:
             story.append(Paragraph(
@@ -611,12 +623,14 @@ class ClassicInvoiceRenderer(BaseReportlabInvoiceRenderer):
                 self.invoice.additional_text,
                 self.stylesheet['Normal']
             ))
-            story.append(Spacer(1, 15 * mm))
+            story.append(Spacer(1, 5 * mm))
 
         tstyledata = [
             ('ALIGN', (1, 0), (-1, -1), 'RIGHT'),
             ('LEFTPADDING', (0, 0), (0, -1), 0),
             ('RIGHTPADDING', (-1, 0), (-1, -1), 0),
+            ('TOPPADDING', (0, 0), (-1, -1), 1),
+            ('BOTTOMPADDING', (0, 0), (-1, -1), 1),
             ('FONTSIZE', (0, 0), (-1, -1), 8),
             ('FONTNAME', (0, 0), (-1, -1), self.font_regular),
         ]

src/pretix/base/management/commands/check_order_transactions.py

@@ -0,0 +1,82 @@
+#
+# This file is part of pretix (Community Edition).
+#
+# Copyright (C) 2014-2020 Raphael Michel and contributors
+# Copyright (C) 2020-2021 rami.io GmbH and contributors
+#
+# This program is free software: you can redistribute it and/or modify it under the terms of the GNU Affero General
+# Public License as published by the Free Software Foundation in version 3 of the License.
+#
+# ADDITIONAL TERMS APPLY: Pursuant to Section 7 of the GNU Affero General Public License, additional terms are
+# applicable granting you additional permissions and placing additional restrictions on your usage of this software.
+# Please refer to the pretix LICENSE file to obtain the full terms applicable to this work. If you did not receive
+# this file, see <https://pretix.eu/about/en/license>.
+#
+# This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied
+# warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU Affero General Public License for more
+# details.
+#
+# You should have received a copy of the GNU Affero General Public License along with this program.  If not, see
+# <https://www.gnu.org/licenses/>.
+#
+from decimal import Decimal
+
+from django.core.management.base import BaseCommand
+from django.db import models
+from django.db.models import Case, F, OuterRef, Q, Subquery, Sum, Value, When
+from django.db.models.functions import Coalesce
+from django_scopes import scopes_disabled
+
+from pretix.base.models import Order, OrderFee, OrderPosition
+from pretix.base.models.orders import Transaction
+
+
+class Command(BaseCommand):
+    help = "Check order for consistency with their transactions"
+
+    @scopes_disabled()
+    def handle(self, *args, **options):
+        qs = Order.objects.annotate(
+            position_total=Coalesce(
+                Subquery(
+                    OrderPosition.objects.filter(
+                        order=OuterRef('pk')
+                    ).order_by().values('order').annotate(p=Sum('price')).values('p'),
+                    output_field=models.DecimalField(decimal_places=2, max_digits=10)
+                ), Value(0), output_field=models.DecimalField(decimal_places=2, max_digits=10)
+            ),
+            fee_total=Coalesce(
+                Subquery(
+                    OrderFee.objects.filter(
+                        order=OuterRef('pk')
+                    ).order_by().values('order').annotate(p=Sum('value')).values('p'),
+                    output_field=models.DecimalField(decimal_places=2, max_digits=10)
+                ), Value(0), output_field=models.DecimalField(decimal_places=2, max_digits=10)
+            ),
+            tx_total=Coalesce(
+                Subquery(
+                    Transaction.objects.filter(
+                        order=OuterRef('pk')
+                    ).order_by().values('order').annotate(p=Sum(F('price') * F('count'))).values('p'),
+                    output_field=models.DecimalField(decimal_places=2, max_digits=10)
+                ), Value(0), output_field=models.DecimalField(decimal_places=2, max_digits=10)
+            ),
+        ).annotate(
+            correct_total=Case(
+                When(Q(status=Order.STATUS_CANCELED) | Q(status=Order.STATUS_EXPIRED) | Q(require_approval=True),
+                     then=Value(0)),
+                default=F('position_total') + F('fee_total'),
+                output_field=models.DecimalField(decimal_places=2, max_digits=10)
+            ),
+        ).exclude(
+            total=F('position_total') + F('fee_total'),
+            tx_total=F('correct_total')
+        ).select_related('event')
+        for o in qs:
+            if abs(o.tx_total - o.correct_total) < Decimal('0.00001') and abs(o.position_total + o.fee_total - o.total) < Decimal('0.00001'):
+                # Ignore SQLite which treats Decimals like floats…
+                continue
+            print(f"Error in order {o.full_code}: status={o.status}, sum(positions)+sum(fees)={o.position_total + o.fee_total}, "
+                  f"order.total={o.total}, sum(transactions)={o.tx_total}, expected={o.correct_total}")
+
+        self.stderr.write(self.style.SUCCESS(f'Check completed.'))

src/pretix/base/management/commands/create_order_transactions.py

@@ -0,0 +1,95 @@
+#
+# This file is part of pretix (Community Edition).
+#
+# Copyright (C) 2014-2020 Raphael Michel and contributors
+# Copyright (C) 2020-2021 rami.io GmbH and contributors
+#
+# This program is free software: you can redistribute it and/or modify it under the terms of the GNU Affero General
+# Public License as published by the Free Software Foundation in version 3 of the License.
+#
+# ADDITIONAL TERMS APPLY: Pursuant to Section 7 of the GNU Affero General Public License, additional terms are
+# applicable granting you additional permissions and placing additional restrictions on your usage of this software.
+# Please refer to the pretix LICENSE file to obtain the full terms applicable to this work. If you did not receive
+# this file, see <https://pretix.eu/about/en/license>.
+#
+# This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied
+# warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU Affero General Public License for more
+# details.
+#
+# You should have received a copy of the GNU Affero General Public License along with this program.  If not, see
+# <https://www.gnu.org/licenses/>.
+#
+import time
+
+from django.core.management.base import BaseCommand
+from django.db.models import F, Max, Q
+from django.utils.timezone import now
+from django_scopes import scopes_disabled
+from tqdm import tqdm
+
+from pretix.base.models import Order
+
+
+class Command(BaseCommand):
+    help = "Create missing order transactions"
+
+    def add_arguments(self, parser):
+        parser.add_argument(
+            "--slowdown",
+            dest="interval",
+            type=int,
+            default=0,
+            help="Interval for staggered execution. If set to a value different then zero, we will "
+                 "wait this many milliseconds between every order we process.",
+        )
+
+    @scopes_disabled()
+    def handle(self, *args, **options):
+        t = 0
+        qs = Order.objects.annotate(
+            last_transaction=Max('transactions__created')
+        ).filter(
+            Q(last_transaction__isnull=True) | Q(last_modified__gt=F('last_transaction')),
+            require_approval=False,
+        ).prefetch_related(
+            'all_positions', 'all_fees'
+        ).order_by(
+            'pk'
+        )
+        last_pk = 0
+        with tqdm(total=qs.count()) as pbar:
+            while True:
+                batch = list(qs.filter(pk__gt=last_pk)[:5000])
+                if not batch:
+                    break
+
+                for o in batch:
+                    if o.last_transaction is None:
+                        tn = o.create_transactions(
+                            positions=o.all_positions.all(),
+                            fees=o.all_fees.all(),
+                            dt_now=o.datetime,
+                            migrated=True,
+                            is_new=True,
+                            _backfill_before_cancellation=True,
+                        )
+                        o.create_transactions(
+                            positions=o.all_positions.all(),
+                            fees=o.all_fees.all(),
+                            dt_now=o.cancellation_date or (o.expires if o.status == Order.STATUS_EXPIRED else o.datetime),
+                            migrated=True,
+                        )
+                    else:
+                        tn = o.create_transactions(
+                            positions=o.all_positions.all(),
+                            fees=o.all_fees.all(),
+                            dt_now=now(),
+                            migrated=True,
+                        )
+                    if tn:
+                        t += 1
+                    time.sleep(0)
+                    pbar.update(1)
+                last_pk = batch[-1].pk
+
+        self.stderr.write(self.style.SUCCESS(f'Created transactions for {t} orders.'))

src/pretix/base/middleware.py

@@ -208,7 +208,7 @@ def _parse_csp(header):
 
 
 def _render_csp(h):
-    return "; ".join(k + ' ' + ' '.join(v) for k, v in h.items())
+    return "; ".join(k + ' ' + ' '.join(v) for k, v in h.items() if v)
 
 
 def _merge_csp(a, b):

src/pretix/base/migrations/0198_invoice_sent_to_customer.py

@@ -0,0 +1,21 @@
+# Generated by Django 3.2.4 on 2021-09-30 10:25
+from datetime import datetime
+
+from django.db import migrations, models
+from pytz import UTC
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('pretixbase', '0197_auto_20210914_0814'),
+    ]
+
+    operations = [
+        migrations.AddField(
+            model_name='invoice',
+            name='sent_to_customer',
+            field=models.DateTimeField(blank=True, null=True, default=UTC.localize(datetime(1970, 1, 1, 0, 0, 0, 0))),
+            preserve_default=False,
+        ),
+    ]

src/pretix/base/migrations/0199_auto_20211005_1050.py

@@ -0,0 +1,23 @@
+# Generated by Django 3.2.4 on 2021-10-05 10:50
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('pretixbase', '0198_invoice_sent_to_customer'),
+    ]
+
+    operations = [
+        migrations.AddField(
+            model_name='item',
+            name='require_membership_hidden',
+            field=models.BooleanField(default=False),
+        ),
+        migrations.AddField(
+            model_name='itemvariation',
+            name='require_membership_hidden',
+            field=models.BooleanField(default=False),
+        ),
+    ]

src/pretix/base/migrations/0200_transaction.py

@@ -0,0 +1,38 @@
+# Generated by Django 3.2.4 on 2021-10-18 10:27
+
+import django.db.models.deletion
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('pretixbase', '0199_auto_20211005_1050'),
+    ]
+
+    operations = [
+        migrations.CreateModel(
+            name='Transaction',
+            fields=[
+                ('id', models.BigAutoField(primary_key=True, serialize=False)),
+                ('created', models.DateTimeField(auto_now_add=True, db_index=True)),
+                ('datetime', models.DateTimeField(db_index=True)),
+                ('migrated', models.BooleanField(default=False)),
+                ('positionid', models.PositiveIntegerField(default=1, null=True)),
+                ('count', models.IntegerField(default=1)),
+                ('price', models.DecimalField(decimal_places=2, max_digits=10)),
+                ('tax_rate', models.DecimalField(decimal_places=2, max_digits=7)),
+                ('tax_value', models.DecimalField(decimal_places=2, max_digits=10)),
+                ('fee_type', models.CharField(max_length=100, null=True)),
+                ('internal_type', models.CharField(max_length=255, null=True)),
+                ('item', models.ForeignKey(null=True, on_delete=django.db.models.deletion.PROTECT, to='pretixbase.item')),
+                ('order', models.ForeignKey(on_delete=django.db.models.deletion.PROTECT, related_name='transactions', to='pretixbase.order')),
+                ('subevent', models.ForeignKey(null=True, on_delete=django.db.models.deletion.PROTECT, to='pretixbase.subevent')),
+                ('tax_rule', models.ForeignKey(null=True, on_delete=django.db.models.deletion.PROTECT, to='pretixbase.taxrule')),
+                ('variation', models.ForeignKey(null=True, on_delete=django.db.models.deletion.PROTECT, to='pretixbase.itemvariation')),
+            ],
+            options={
+                'ordering': ('datetime', 'pk'),
+            },
+        ),
+    ]

src/pretix/base/migrations/0201_invoiceline_event_location.py

@@ -0,0 +1,18 @@
+# Generated by Django 3.2.4 on 2021-11-03 09:24
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('pretixbase', '0200_transaction'),
+    ]
+
+    operations = [
+        migrations.AddField(
+            model_name='invoiceline',
+            name='event_location',
+            field=models.TextField(null=True),
+        ),
+    ]

src/pretix/base/migrations/0202_user_needs_password_change.py

@@ -0,0 +1,18 @@
+# Generated by Django 3.2.9 on 2021-11-04 13:05
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('pretixbase', '0201_invoiceline_event_location'),
+    ]
+
+    operations = [
+        migrations.AddField(
+            model_name='user',
+            name='needs_password_change',
+            field=models.BooleanField(default=False),
+        ),
+    ]

src/pretix/base/migrations/0203_orderposition_is_bundled.py

@@ -0,0 +1,17 @@
+# Generated by Django 3.2.2 on 2021-11-08 07:51
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+    dependencies = [
+        ('pretixbase', '0202_user_needs_password_change'),
+    ]
+
+    operations = [
+        migrations.AddField(
+            model_name='orderposition',
+            name='is_bundled',
+            field=models.BooleanField(default=False),
+        ),
+    ]

src/pretix/base/migrations/0204_orderposition_backfill_is_bundled.py

@@ -0,0 +1,46 @@
+# Generated by Django 3.2.2 on 2021-11-08 07:51
+
+from django.db import migrations, models
+from django.db.models import Count, OuterRef, Subquery
+from django.db.models.functions import Coalesce
+
+
+def fill_is_bundled(apps, schema_editor):
+    # We cannot really know if a position was bundled or an add-on, but we can at least guess
+    ItemBundle = apps.get_model("pretixbase", "ItemBundle")
+    OrderPosition = apps.get_model("pretixbase", "OrderPosition")
+
+    for ib in ItemBundle.objects.iterator():
+        OrderPosition.all.alias(
+            pos_earlier=Coalesce(Subquery(
+                OrderPosition.all.filter(
+                    canceled=False,
+                    addon_to=OuterRef('addon_to'),
+                    item=ib.bundled_item,
+                    variation=ib.bundled_variation,
+                    positionid__lt=OuterRef('positionid'),
+                ).values('addon_to').order_by().annotate(c=Count('*')).values('c'),
+                output_field=models.IntegerField()
+            ), 0)
+        ).filter(
+            canceled=False,
+            addon_to__item=ib.base_item,
+            item=ib.bundled_item,
+            variation=ib.bundled_variation,
+            pos_earlier__lt=ib.count,
+        ).update(
+            is_bundled=True
+        )
+
+
+class Migration(migrations.Migration):
+    dependencies = [
+        ('pretixbase', '0203_orderposition_is_bundled'),
+    ]
+
+    operations = [
+        migrations.RunPython(
+            fill_is_bundled,
+            migrations.RunPython.noop,
+        ),
+    ]

src/pretix/base/models/__init__.py

@@ -42,8 +42,9 @@ from .notifications import NotificationSetting
 from .orders import (
     AbstractPosition, CachedCombinedTicket, CachedTicket, CartPosition,
     InvoiceAddress, Order, OrderFee, OrderPayment, OrderPosition, OrderRefund,
-    QuestionAnswer, RevokedTicketSecret, cachedcombinedticket_name,
-    cachedticket_name, generate_position_secret, generate_secret,
+    QuestionAnswer, RevokedTicketSecret, Transaction,
+    cachedcombinedticket_name, cachedticket_name, generate_position_secret,
+    generate_secret,
 )
 from .organizer import (
     Organizer, Organizer_SettingsStore, Team, TeamAPIToken, TeamInvite,

src/pretix/base/models/_transactions.py

@@ -0,0 +1,113 @@
+#
+# This file is part of pretix (Community Edition).
+#
+# Copyright (C) 2014-2020 Raphael Michel and contributors
+# Copyright (C) 2020-2021 rami.io GmbH and contributors
+#
+# This program is free software: you can redistribute it and/or modify it under the terms of the GNU Affero General
+# Public License as published by the Free Software Foundation in version 3 of the License.
+#
+# ADDITIONAL TERMS APPLY: Pursuant to Section 7 of the GNU Affero General Public License, additional terms are
+# applicable granting you additional permissions and placing additional restrictions on your usage of this software.
+# Please refer to the pretix LICENSE file to obtain the full terms applicable to this work. If you did not receive
+# this file, see <https://pretix.eu/about/en/license>.
+#
+# This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied
+# warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU Affero General Public License for more
+# details.
+#
+# You should have received a copy of the GNU Affero General Public License along with this program.  If not, see
+# <https://www.gnu.org/licenses/>.
+#
+
+"""
+This module contains helper functions that are supposed to call out code paths missing calls to
+``Order.create_transaction()`` by actively breaking them. Read the docstring of the ``Transaction`` class for a
+detailed reasoning why this exists.
+"""
+import inspect
+import logging
+import os
+import threading
+
+from django.conf import settings
+from django.db import transaction
+
+dirty_transactions = threading.local()
+
+logger = logging.getLogger(__name__)
+fail_loudly = os.getenv('PRETIX_DIRTY_TRANSACTIONS_QUIET', 'false' if settings.DEBUG else 'true') not in ('true', 'True', 'on', '1')
+
+
+class DirtyTransactionsForOrderException(Exception):
+    pass
+
+
+def _fail(message):
+    if fail_loudly:
+        raise DirtyTransactionsForOrderException(message)
+    else:
+        if settings.SENTRY_ENABLED:
+            import sentry_sdk
+
+            sentry_sdk.capture_message(message, "fatal")
+
+        logger.warning(message, stack_info=True)
+
+
+def _check_for_dirty_orders():
+    if getattr(dirty_transactions, 'order_ids', None) is None:
+        dirty_transactions.order_ids = set()
+    try:
+        if dirty_transactions.order_ids and dirty_transactions.order_ids != {None}:
+            _fail(
+                f"In the transaction that just ended, you created or modified an Order, OrderPosition, or OrderFee "
+                f"object in a way that you should have called `order.create_transactions()` afterwards. The transaction "
+                f"still went through and your data can be fixed with the `create_order_transactions` management command "
+                f"but you should update your code to prevent this from happening. Affected order IDs: {dirty_transactions.order_ids}"
+            )
+    finally:
+        dirty_transactions.order_ids.clear()
+
+
+def _transactions_mark_order_dirty(order_id, using=None):
+    if "PYTEST_CURRENT_TEST" in os.environ:
+        # We don't care about Order.objects.create() calls in test code so let's try to figure out if this is test code
+        # or not.
+        for frame in inspect.stack():
+            if 'pretix/base/models/orders' in frame.filename:
+                continue
+            elif 'test_' in frame.filename or 'conftest.py in frame.filename':
+                return
+            elif 'pretix/' in frame.filename or 'pretix_' in frame.filename:
+                # This went through non-test code, let's consider it non-test
+                break
+
+    if order_id is None:
+        return
+
+    conn = transaction.get_connection(using)
+    if not conn.in_atomic_block:
+        _fail(
+            "You modified an Order, OrderPosition, or OrderFee object in a way that should create "
+            "a new Transaction object within the same database transaction, however you are not "
+            "doing it inside a database transaction!"
+        )
+
+    if getattr(dirty_transactions, 'order_ids', None) is None:
+        dirty_transactions.order_ids = set()
+
+    if _check_for_dirty_orders not in [func for savepoint_id, func in conn.run_on_commit]:
+        transaction.on_commit(_check_for_dirty_orders, using)
+        dirty_transactions.order_ids.clear()  # This is necessary to clean up after old threads with rollbacked transactions
+
+    dirty_transactions.order_ids.add(order_id)
+
+
+def _transactions_mark_order_clean(order_id):
+    if getattr(dirty_transactions, 'order_ids', None) is None:
+        dirty_transactions.order_ids = set()
+    try:
+        dirty_transactions.order_ids.remove(order_id)
+    except KeyError:
+        pass

src/pretix/base/models/auth.py

@@ -113,6 +113,8 @@ class User(AbstractBaseUser, PermissionsMixin, LoggingMixin):
     :type date_joined: datetime
     :param locale: The user's preferred locale code.
     :type locale: str
+    :param needs_password_change: Whether this user's password needs to be changed.
+    :type needs_password_change: bool
     :param timezone: The user's preferred timezone.
     :type timezone: str
     """
@@ -130,6 +132,8 @@ class User(AbstractBaseUser, PermissionsMixin, LoggingMixin):
                                    verbose_name=_('Is site admin'))
     date_joined = models.DateTimeField(auto_now_add=True,
                                        verbose_name=_('Date joined'))
+    needs_password_change = models.BooleanField(default=False,
+                                                verbose_name=_('Force user to select a new password'))
     locale = models.CharField(max_length=50,
                               choices=settings.LANGUAGES,
                               default=settings.LANGUAGE_CODE,

src/pretix/base/models/customers.py

@@ -25,6 +25,7 @@ from django.contrib.auth.hashers import (
     check_password, is_password_usable, make_password,
 )
 from django.db import models
+from django.db.models import F, Q
 from django.utils.crypto import get_random_string, salted_hmac
 from django.utils.translation import gettext_lazy as _, pgettext_lazy
 from django_scopes import ScopedManager, scopes_disabled
@@ -183,6 +184,12 @@ class Customer(LoggedModel):
     def stored_addresses(self):
         return self.invoice_addresses(manager='profiles')
 
+    def usable_memberships(self, for_event, testmode=False):
+        return self.memberships.active(for_event).with_usages().filter(
+            Q(membership_type__max_usages__isnull=True) | Q(usages__lt=F('membership_type__max_usages')),
+            testmode=testmode,
+        )
+
 
 class AttendeeProfile(models.Model):
     customer = models.ForeignKey(

src/pretix/base/models/event.py

@@ -57,6 +57,7 @@ from django.urls import reverse
 from django.utils.crypto import get_random_string
 from django.utils.formats import date_format
 from django.utils.functional import cached_property
+from django.utils.html import format_html
 from django.utils.timezone import make_aware, now
 from django.utils.translation import gettext, gettext_lazy as _
 from django_scopes import ScopedManager, scopes_disabled
@@ -145,7 +146,7 @@ class EventMixin:
             ("SHORT_" if short else "") + ("DATETIME_FORMAT" if self.settings.show_times and show_times else "DATE_FORMAT")
         )
 
-    def get_date_range_display(self, tz=None, force_show_end=False) -> str:
+    def get_date_range_display(self, tz=None, force_show_end=False, as_html=False) -> str:
         """
         Returns a formatted string containing the start date and the end date
         of the event with respect to the current locale and to the ``show_date_to``
@@ -153,8 +154,17 @@ class EventMixin:
         """
         tz = tz or self.timezone
         if (not self.settings.show_date_to and not force_show_end) or not self.date_to:
+            if as_html:
+                return format_html(
+                    "<time datetime=\"{}\">{}</time>",
+                    _date(self.date_from.astimezone(tz), "Y-m-d"),
+                    _date(self.date_from.astimezone(tz), "DATE_FORMAT"),
+                )
             return _date(self.date_from.astimezone(tz), "DATE_FORMAT")
-        return daterange(self.date_from.astimezone(tz), self.date_to.astimezone(tz))
+        return daterange(self.date_from.astimezone(tz), self.date_to.astimezone(tz), as_html)
+
+    def get_date_range_display_as_html(self, tz=None, force_show_end=False) -> str:
+        return self.get_date_range_display(tz, force_show_end, as_html=True)
 
     def get_time_range_display(self, tz=None, force_show_end=False) -> str:
         """
@@ -555,6 +565,8 @@ class Event(EventMixin, LoggedModel):
         self.settings.ticketoutput_pdf__enabled = True
         self.settings.ticketoutput_passbook__enabled = True
         self.settings.event_list_type = 'calendar'
+        self.settings.invoice_email_attachment = True
+        self.settings.name_scheme = 'given_family'
 
     @property
     def social_image(self):
@@ -1421,7 +1433,7 @@ class SubEvent(EventMixin, LoggedModel):
         return self.event.currency
 
     def allow_delete(self):
-        return not self.orderposition_set.exists()
+        return not self.orderposition_set.exists() and not self.transaction_set.exists()
 
     def delete(self, *args, **kwargs):
         clear_cache = kwargs.pop('clear_cache', False)

src/pretix/base/models/invoices.py

@@ -159,6 +159,8 @@ class Invoice(models.Model):
     # False: The invoice wasn't sent and never will, because sending was not configured at the time of the check.
     sent_to_organizer = models.BooleanField(null=True, blank=True)
 
+    sent_to_customer = models.DateTimeField(null=True, blank=True)
+
     file = models.FileField(null=True, blank=True, upload_to=invoice_filename, max_length=255)
 
     objects = ScopedManager(organizer='event__organizer')
@@ -235,7 +237,7 @@ class Invoice(models.Model):
     def _get_invoice_number_from_order(self):
         return '{order}-{count}'.format(
             order=self.order.code,
-            count=Invoice.objects.filter(event=self.event, order=self.order).count() + 1,
+            count=Invoice.objects.filter(event=self.event, prefix=self.prefix, invoice_no__startswith=f"{self.order.code}-", order=self.order).count() + 1,
         )
 
     def save(self, *args, **kwargs):
@@ -262,6 +264,7 @@ class Invoice(models.Model):
                     self.invoice_no = self._get_invoice_number_from_order()
                 try:
                     with transaction.atomic():
+                        self.full_invoice_no = self.prefix + self.invoice_no
                         return super().save(*args, **kwargs)
                 except DatabaseError:
                     # Suppress duplicate key errors and try again
@@ -300,6 +303,9 @@ class Invoice(models.Model):
     def __repr__(self):
         return '<Invoice {} / {}>'.format(self.full_invoice_no, self.pk)
 
+    def __str__(self):
+        return self.full_invoice_no
+
 
 class InvoiceLine(models.Model):
     """
@@ -323,6 +329,8 @@ class InvoiceLine(models.Model):
     :type event_date_from: datetime
     :param event_date_to: Event end date of the (sub)event at the time the invoice was created
     :type event_date_to: datetime
+    :param event_location: Event location of the (sub)event at the time the invoice was created
+    :type event_location: str
     :param item: The item this line refers to
     :type item: Item
     :param variation: The variation this line refers to
@@ -340,6 +348,7 @@ class InvoiceLine(models.Model):
     subevent = models.ForeignKey('SubEvent', null=True, blank=True, on_delete=models.PROTECT)
     event_date_from = models.DateTimeField(null=True)
     event_date_to = models.DateTimeField(null=True)
+    event_location = models.TextField(null=True, blank=True)
     item = models.ForeignKey('Item', null=True, blank=True, on_delete=models.PROTECT)
     variation = models.ForeignKey('ItemVariation', null=True, blank=True, on_delete=models.PROTECT)
     attendee_name = models.TextField(null=True, blank=True)

src/pretix/base/models/items.py

@@ -523,6 +523,12 @@ class Item(LoggedModel):
         verbose_name=_('Allowed membership types'),
         blank=True,
     )
+    require_membership_hidden = models.BooleanField(
+        verbose_name=_('Hide without a valid membership'),
+        help_text=_('Do not show this unless the customer is logged in and has a valid membership. Be aware that '
+                    'this means it will never be visible in the widget.'),
+        default=False,
+    )
     grant_membership_type = models.ForeignKey(
         'MembershipType',
         null=True, blank=True,
@@ -687,9 +693,9 @@ class Item(LoggedModel):
         return res
 
     def allow_delete(self):
-        from pretix.base.models.orders import OrderPosition
+        from pretix.base.models.orders import OrderPosition, Transaction
 
-        return not OrderPosition.all.filter(item=self).exists()
+        return not Transaction.objects.filter(item=self).exists() and not OrderPosition.all.filter(item=self).exists()
 
     @property
     def includes_mixed_tax_rate(self):
@@ -802,6 +808,12 @@ class ItemVariation(models.Model):
         verbose_name=_('Membership types'),
         blank=True,
     )
+    require_membership_hidden = models.BooleanField(
+        verbose_name=_('Hide without a valid membership'),
+        help_text=_('Do not show this unless the customer is logged in and has a valid membership. Be aware that '
+                    'this means it will never be visible in the widget.'),
+        default=False,
+    )
     available_from = models.DateTimeField(
         verbose_name=_("Available from"),
         null=True, blank=True,
@@ -946,10 +958,13 @@ class ItemVariation(models.Model):
         return self.position < other.position
 
     def allow_delete(self):
-        from pretix.base.models.orders import CartPosition, OrderPosition
+        from pretix.base.models.orders import (
+            CartPosition, OrderPosition, Transaction,
+        )
 
         return (
-            not OrderPosition.objects.filter(variation=self).exists()
+            not Transaction.objects.filter(variation=self).exists()
+            and not OrderPosition.objects.filter(variation=self).exists()
             and not CartPosition.objects.filter(variation=self).exists()
         )
 

src/pretix/base/models/memberships.py

@@ -95,6 +95,7 @@ class MembershipQuerySet(models.QuerySet):
 
     def active(self, ev):
         return self.filter(
+            canceled=False,
             date_start__lte=ev.date_from,
             date_end__gte=ev.date_from
         )
@@ -175,7 +176,7 @@ class Membership(models.Model):
         else:
             dt = now()
 
-        return dt >= self.date_start and dt <= self.date_end
+        return not self.canceled and dt >= self.date_start and dt <= self.date_end
 
     def allow_delete(self):
         return self.testmode and not self.orderposition_set.exists()

src/pretix/base/models/orders.py

@@ -75,11 +75,14 @@ from pretix.base.email import get_email_context
 from pretix.base.i18n import language
 from pretix.base.models import Customer, User
 from pretix.base.reldate import RelativeDateWrapper
-from pretix.base.services.locking import NoLockManager
+from pretix.base.services.locking import LOCK_TIMEOUT, NoLockManager
 from pretix.base.settings import PERSON_NAME_SCHEMES
 from pretix.base.signals import order_gracefully_delete
 
 from ...helpers.countries import CachedCountries, FastCountryField
+from ._transactions import (
+    _fail, _transactions_mark_order_clean, _transactions_mark_order_dirty,
+)
 from .base import LockModel, LoggedModel
 from .event import Event, SubEvent
 from .items import Item, ItemVariation, Question, QuestionOption, Quota
@@ -262,6 +265,11 @@ class Order(LockModel, LoggedModel):
     def __str__(self):
         return self.full_code
 
+    def __init__(self, *args, **kwargs):
+        super().__init__(*args, **kwargs)
+        if 'require_approval' not in self.get_deferred_fields() and 'status' not in self.get_deferred_fields():
+            self.__initial_status_paid_or_pending = self.status in (Order.STATUS_PENDING, Order.STATUS_PAID) and not self.require_approval
+
     def gracefully_delete(self, user=None, auth=None):
         from . import GiftCard, GiftCardTransaction, Membership, Voucher
 
@@ -289,6 +297,7 @@ class Order(LockModel, LoggedModel):
         OrderPosition.all.filter(order=self, addon_to__isnull=False).delete()
         OrderPosition.all.filter(order=self).delete()
         OrderFee.all.filter(order=self).delete()
+        Transaction.objects.filter(order=self).delete()
         self.refunds.all().delete()
         self.payments.all().delete()
         self.event.cache.delete('complain_testmode_orders')
@@ -444,7 +453,27 @@ class Order(LockModel, LoggedModel):
             self.datetime = now()
         if not self.expires:
             self.set_expires()
-        super().save(**kwargs)
+
+        is_new = not self.pk
+        update_fields = kwargs.get('update_fields', [])
+        if 'require_approval' not in self.get_deferred_fields() and 'status' not in self.get_deferred_fields():
+            status_paid_or_pending = self.status in (Order.STATUS_PENDING, Order.STATUS_PAID) and not self.require_approval
+            if status_paid_or_pending != self.__initial_status_paid_or_pending:
+                _transactions_mark_order_dirty(self.pk, using=kwargs.get('using', None))
+        elif (
+            not kwargs.get('force_save_with_deferred_fields', None) and
+            (not update_fields or ('require_approval' not in update_fields and 'status' not in update_fields))
+        ):
+            _fail("It is unsafe to call save() on an OrderFee with deferred fields since we can't check if you missed "
+                  "creating a transaction. Call save(force_save_with_deferred_fields=True) if you really want to do "
+                  "this.")
+
+        r = super().save(**kwargs)
+
+        if is_new:
+            _transactions_mark_order_dirty(self.pk, using=kwargs.get('using', None))
+
+        return r
 
     def touch(self):
         self.save(update_fields=['last_modified'])
@@ -552,6 +581,7 @@ class Order(LockModel, LoggedModel):
         Returns whether or not this order can be canceled by the user.
         """
         from .checkin import Checkin
+        from .items import ItemAddOn
 
         if self.status not in (Order.STATUS_PENDING, Order.STATUS_PAID) or not self.count_positions:
             return False
@@ -577,7 +607,10 @@ class Order(LockModel, LoggedModel):
         if self.user_change_deadline and now() > self.user_change_deadline:
             return False
 
-        return self.event.settings.change_allow_user_variation and any([op.has_variations for op in positions])
+        return (
+            (self.event.settings.change_allow_user_variation and any([op.has_variations for op in positions])) or
+            (self.event.settings.change_allow_user_addons and ItemAddOn.objects.filter(base_item_id__in=[op.item_id for op in positions]).exists())
+        )
 
     @property
     @scopes_disabled()
@@ -999,6 +1032,59 @@ class Order(LockModel, LoggedModel):
                 continue
             yield op
 
+    def create_transactions(self, is_new=False, positions=None, fees=None, dt_now=None, migrated=False,
+                            _backfill_before_cancellation=False, save=True):
+        dt_now = dt_now or now()
+
+        # Count the transactions we already have
+        current_transaction_count = Counter()
+        if not is_new:
+            for t in Transaction.objects.filter(order=self):  # do not use related manager, we want to avoid cached data
+                current_transaction_count[Transaction.key(t)] += t.count
+
+        # Count the transactions we'd actually need
+        target_transaction_count = Counter()
+        if (_backfill_before_cancellation or self.status in (Order.STATUS_PENDING, Order.STATUS_PAID)) and not self.require_approval:
+            positions = self.positions.all() if positions is None else positions
+            for p in positions:
+                if p.canceled and not _backfill_before_cancellation:
+                    continue
+                target_transaction_count[Transaction.key(p)] += 1
+
+            fees = self.fees.all() if fees is None else fees
+            for f in fees:
+                if f.canceled and not _backfill_before_cancellation:
+                    continue
+                target_transaction_count[Transaction.key(f)] += 1
+
+        keys = set(target_transaction_count.keys()) | set(current_transaction_count.keys())
+        create = []
+        for k in keys:
+            positionid, itemid, variationid, subeventid, price, taxrate, taxruleid, taxvalue, feetype, internaltype = k
+            d = target_transaction_count[k] - current_transaction_count[k]
+            if d:
+                create.append(Transaction(
+                    order=self,
+                    datetime=dt_now,
+                    migrated=migrated,
+                    positionid=positionid,
+                    count=d,
+                    item_id=itemid,
+                    variation_id=variationid,
+                    subevent_id=subeventid,
+                    price=price,
+                    tax_rate=taxrate,
+                    tax_rule_id=taxruleid,
+                    tax_value=taxvalue,
+                    fee_type=feetype,
+                    internal_type=internaltype,
+                ))
+        create.sort(key=lambda t: (0 if t.count < 0 else 1, t.positionid or 0))
+        if save:
+            Transaction.objects.bulk_create(create)
+        _transactions_mark_order_clean(self.pk)
+        return create
+
 
 def answerfile_name(instance, filename: str) -> str:
     secret = get_random_string(length=32, allowed_chars=string.ascii_letters + string.digits)
@@ -1224,6 +1310,7 @@ class AbstractPosition(models.Model):
     seat = models.ForeignKey(
         'Seat', null=True, blank=True, on_delete=models.PROTECT
     )
+    is_bundled = models.BooleanField(default=False)
 
     company = models.CharField(max_length=255, blank=True, verbose_name=_('Company name'), null=True)
     street = models.TextField(verbose_name=_('Address'), blank=True, null=True)
@@ -1460,7 +1547,7 @@ class OrderPayment(models.Model):
         return self.order.event.get_payment_providers(cached=True).get(self.provider)
 
     @transaction.atomic()
-    def _mark_paid(self, force, count_waitinglist, user, auth, ignore_date=False, overpaid=False):
+    def _mark_paid_inner(self, force, count_waitinglist, user, auth, ignore_date=False, overpaid=False):
         from pretix.base.signals import order_paid
         can_be_paid = self.order._can_be_paid(count_waitinglist=count_waitinglist, ignore_date=ignore_date, force=force)
         if can_be_paid is not True:
@@ -1468,6 +1555,7 @@ class OrderPayment(models.Model):
                 'message': can_be_paid
             }, user=user, auth=auth)
             raise Quota.QuotaExceededException(can_be_paid)
+        status_change = self.order.status != Order.STATUS_PENDING
         self.order.status = Order.STATUS_PAID
         self.order.save(update_fields=['status'])
 
@@ -1481,6 +1569,8 @@ class OrderPayment(models.Model):
         if overpaid:
             self.order.log_action('pretix.event.order.overpaid', {}, user=user, auth=auth)
         order_paid.send(self.order.event, order=self.order)
+        if status_change:
+            self.order.create_transactions()
 
     def fail(self, info=None, user=None, auth=None):
         """
@@ -1533,10 +1623,6 @@ class OrderPayment(models.Model):
         :type mail_text: str
         :raises Quota.QuotaExceededException: if the quota is exceeded and ``force`` is ``False``
         """
-        from pretix.base.services.invoices import (
-            generate_invoice, invoice_qualified,
-        )
-
         with transaction.atomic():
             locked_instance = OrderPayment.objects.select_for_update().get(pk=self.pk)
             if locked_instance.state == self.PAYMENT_STATE_CONFIRMED:
@@ -1580,7 +1666,15 @@ class OrderPayment(models.Model):
             ))
             return
 
-        if (self.order.status == Order.STATUS_PENDING and self.order.expires > now() + timedelta(hours=12)) or not lock:
+        self._mark_order_paid(count_waitinglist, send_mail, force, user, auth, mail_text, ignore_date, lock, payment_sum - refund_sum)
+
+    def _mark_order_paid(self, count_waitinglist=True, send_mail=True, force=False, user=None, auth=None, mail_text='',
+                         ignore_date=False, lock=True, payment_refund_sum=0):
+        from pretix.base.services.invoices import (
+            generate_invoice, invoice_qualified,
+        )
+
+        if (self.order.status == Order.STATUS_PENDING and self.order.expires > now() + timedelta(seconds=LOCK_TIMEOUT * 2)) or not lock:
             # Performance optimization. In this case, there's really no reason to lock everything and an atomic
             # database transaction is more than enough.
             lockfn = NoLockManager
@@ -1588,8 +1682,8 @@ class OrderPayment(models.Model):
             lockfn = self.order.event.lock
 
         with lockfn():
-            self._mark_paid(force, count_waitinglist, user, auth, overpaid=payment_sum - refund_sum > self.order.total,
-                            ignore_date=ignore_date)
+            self._mark_paid_inner(force, count_waitinglist, user, auth, overpaid=payment_refund_sum > self.order.total,
+                                  ignore_date=ignore_date)
 
         invoice = None
         if invoice_qualified(self.order):
@@ -1958,6 +2052,12 @@ class OrderFee(models.Model):
     def net_value(self):
         return self.value - self.tax_value
 
+    def __init__(self, *args, **kwargs):
+        super().__init__(*args, **kwargs)
+        if not self.get_deferred_fields():
+            self.__initial_transaction_key = Transaction.key(self)
+            self.__initial_canceled = self.canceled
+
     def __str__(self):
         if self.description:
             return '{} - {}'.format(self.get_fee_type_display(), self.description)
@@ -1996,6 +2096,15 @@ class OrderFee(models.Model):
         if self.tax_rate is None:
             self._calculate_tax()
         self.order.touch()
+
+        if not self.get_deferred_fields():
+            if Transaction.key(self) != self.__initial_transaction_key or self.canceled != self.__initial_canceled or not self.pk:
+                _transactions_mark_order_dirty(self.order_id, using=kwargs.get('using', None))
+        elif not kwargs.get('force_save_with_deferred_fields', None):
+            _fail("It is unsafe to call save() on an OrderFee with deferred fields since we can't check if you missed "
+                  "creating a transaction. Call save(force_save_with_deferred_fields=True) if you really want to do "
+                  "this.")
+
         return super().save(*args, **kwargs)
 
     def delete(self, **kwargs):
@@ -2010,7 +2119,7 @@ class OrderPosition(AbstractPosition):
     AbstractPosition.
 
     The default ``OrderPosition.objects`` manager only contains fees that are not ``canceled``. If
-    you ant all objects, you need to use ``OrderPosition.all`` instead.
+    you want all objects, you need to use ``OrderPosition.all`` instead.
 
     :param order: The order this position is a part of
     :type order: Order
@@ -2061,6 +2170,12 @@ class OrderPosition(AbstractPosition):
     all = ScopedManager(organizer='order__event__organizer')
     objects = ActivePositionManager()
 
+    def __init__(self, *args, **kwargs):
+        super().__init__(*args, **kwargs)
+        if not self.get_deferred_fields():
+            self.__initial_transaction_key = Transaction.key(self)
+            self.__initial_canceled = self.canceled
+
     class Meta:
         verbose_name = _("Order position")
         verbose_name_plural = _("Order positions")
@@ -2104,6 +2219,7 @@ class OrderPosition(AbstractPosition):
             op._calculate_tax()
             op.positionid = i + 1
             op.save()
+            ops.append(op)
             cp_mapping[cartpos.pk] = op
             for answ in cartpos.answers.all():
                 answ.orderposition = op
@@ -2169,6 +2285,14 @@ class OrderPosition(AbstractPosition):
         if not self.pseudonymization_id:
             self.assign_pseudonymization_id()
 
+        if not self.get_deferred_fields():
+            if Transaction.key(self) != self.__initial_transaction_key or self.canceled != self.__initial_canceled or not self.pk:
+                _transactions_mark_order_dirty(self.order_id, using=kwargs.get('using', None))
+        elif not kwargs.get('force_save_with_deferred_fields', None):
+            _fail("It is unsafe to call save() on an OrderFee with deferred fields since we can't check if you missed "
+                  "creating a transaction. Call save(force_save_with_deferred_fields=True) if you really want to do "
+                  "this.")
+
         return super().save(*args, **kwargs)
 
     @scopes_disabled()
@@ -2212,7 +2336,7 @@ class OrderPosition(AbstractPosition):
         :param attach_ical: Attach relevant ICS files
         """
         from pretix.base.services.mail import (
-            SendMailException, mail, render_mail,
+            SendMailException, TolerantDict, mail, render_mail,
         )
 
         if not self.attendee_email:
@@ -2225,6 +2349,7 @@ class OrderPosition(AbstractPosition):
             recipient = self.attendee_email
             try:
                 email_content = render_mail(template, context)
+                subject = str(subject).format_map(TolerantDict(context))
                 mail(
                     recipient, subject, template, context,
                     self.event, self.order.locale, order=self.order, headers=headers, sender=sender,
@@ -2263,6 +2388,151 @@ class OrderPosition(AbstractPosition):
             )
 
 
+class Transaction(models.Model):
+    """
+    Transactions are a data structure that is redundant on the first sight but makes it possible to create good
+    financial reporting.
+
+    To understand this, think of "orders" as something like a contractual relationship between the organizer and the
+    customer which requires to customer to pay some money and the organizer to provide a ticket.
+
+    The ``Order``, ``OrderPosition``, and ``OrderFee`` models combined give a representation of the current contractual
+    status of this relationship, i.e. how much and what is owed. The ``OrderPayment`` and ``OrderRefund`` models indicate
+    the "other side" of the relationship, i.e. how much of the financial obligation has been met so far.
+
+    However, while ``OrderPayment`` and ``OrderRefund`` objects are "final" and no longer change once they reached their
+    final state, ``Order``, ``OrderPosition`` and ``OrderFee`` are highly mutable and can change at any time, e.g. if
+    the customer moves their booking to a different day or a discount is applied retroactively.
+
+    Therefore those models can be used to answer the question "how many tickets of type X have been sold for my event
+    as of today?" but they cannot accurately answer the question "how many tickets of type X have been sold for my event
+    as of last month?" because they lack this kind of historical information.
+
+    Transactions help here because they are "immutable copies" or "modification records" of call positions and fees
+    at the time of their creation and change. They only record data that is usually relevant for financial reporting,
+    such as amounts, prices, products and dates involved. They do not record data like attendee names etc.
+
+    Even before the introduction of the Transaction Model pretix *did* store historical data for auditability in the
+    LogEntry model. However, it's almost impossible to do efficient reporting on that data.
+
+    Transactions should never be generated manually but only through the ``order.create_transactions()``
+    method which should be called **within the same database transaction**.
+
+    The big downside of this approach is that you need to remember to update transaction records every time you change
+    or create orders in new code paths. The mechanism introduced in ``pretix.base.models._transactions`` as well as
+    the ``save()`` methods of ``Order``, ``OrderPosition`` and ``OrderFee`` intends to help you notice if you missed
+    it. The only thing this *doesn't* catch is usage of ``OrderPosition.objects.bulk_create`` (and likewise for
+    ``bulk_update`` and ``OrderFee``).
+
+    :param id: ID of the transaction
+    :param order: Order the transaction belongs to
+    :param datetime: Date and time of the transaction
+    :param migrated: Whether this object was reconstructed because the order was created before transactions where introduced
+    :param positionid: Affected Position ID, in case this transaction represents a change in an order position
+    :param count: An amount, multiplicator for price etc. For order positions this can *currently* only be -1 or +1, for
+                  fees it can also be more in special cases
+    :param item: ``Item``, in case this transaction represents a change in an order position
+    :param variation: ``ItemVariation``, in case this transaction represents a change in an order position
+    :param subevent: ``subevent``, in case this transaction represents a change in an order position
+    :param price: Price of the changed position
+    :param tax_rate: Tax rate of the changed position
+    :param tax_rule: Used tax rule
+    :param tax_value: Tax value in event currency
+    :param fee_type: Fee type code in case this transaction represents a change in an order fee
+    :param internal_type: Internal fee type in case this transaction represents a change in an order fee
+    """
+    id = models.BigAutoField(primary_key=True)
+    order = models.ForeignKey(
+        Order,
+        verbose_name=_("Order"),
+        related_name='transactions',
+        on_delete=models.PROTECT
+    )
+    created = models.DateTimeField(
+        auto_now_add=True,
+        db_index=True,
+    )
+    datetime = models.DateTimeField(
+        verbose_name=_("Date"),
+        db_index=True,
+    )
+    migrated = models.BooleanField(
+        default=False
+    )
+    positionid = models.PositiveIntegerField(default=1, null=True, blank=True)
+    count = models.IntegerField(
+        default=1
+    )
+    item = models.ForeignKey(
+        Item,
+        null=True, blank=True,
+        verbose_name=_("Item"),
+        on_delete=models.PROTECT
+    )
+    variation = models.ForeignKey(
+        ItemVariation,
+        null=True, blank=True,
+        verbose_name=_("Variation"),
+        on_delete=models.PROTECT
+    )
+    subevent = models.ForeignKey(
+        SubEvent,
+        null=True, blank=True,
+        on_delete=models.PROTECT,
+        verbose_name=pgettext_lazy("subevent", "Date"),
+    )
+    price = models.DecimalField(
+        decimal_places=2, max_digits=10,
+        verbose_name=_("Price")
+    )
+    tax_rate = models.DecimalField(
+        max_digits=7, decimal_places=2,
+        verbose_name=_('Tax rate')
+    )
+    tax_rule = models.ForeignKey(
+        'TaxRule',
+        on_delete=models.PROTECT,
+        null=True, blank=True
+    )
+    tax_value = models.DecimalField(
+        max_digits=10, decimal_places=2,
+        verbose_name=_('Tax value')
+    )
+    fee_type = models.CharField(
+        max_length=100, choices=OrderFee.FEE_TYPES, null=True, blank=True
+    )
+    internal_type = models.CharField(max_length=255, null=True, blank=True)
+
+    class Meta:
+        ordering = 'datetime', 'pk'
+
+    def save(self, *args, **kwargs):
+        if not self.fee_type and not self.item:
+            raise ValidationError('Should set either item or fee type')
+        return super().save(*args, **kwargs)
+
+    @staticmethod
+    def key(obj):
+        if isinstance(obj, Transaction):
+            return (obj.positionid, obj.item_id, obj.variation_id, obj.subevent_id, obj.price, obj.tax_rate,
+                    obj.tax_rule_id, obj.tax_value, obj.fee_type, obj.internal_type)
+        elif isinstance(obj, OrderPosition):
+            return (obj.positionid, obj.item_id, obj.variation_id, obj.subevent_id, obj.price, obj.tax_rate,
+                    obj.tax_rule_id, obj.tax_value, None, None)
+        elif isinstance(obj, OrderFee):
+            return (None, None, None, None, obj.value, obj.tax_rate,
+                    obj.tax_rule_id, obj.tax_value, obj.fee_type, obj.internal_type)
+        raise ValueError('invalid state')  # noqa
+
+    @property
+    def full_price(self):
+        return self.price * self.count
+
+    @property
+    def full_tax_value(self):
+        return self.tax_value * self.count
+
+
 class CartPosition(AbstractPosition):
     """
     A cart position is similar to an order line, except that it is not
@@ -2301,7 +2571,6 @@ class CartPosition(AbstractPosition):
         max_digits=10, decimal_places=2,
         null=True, blank=True
     )
-    is_bundled = models.BooleanField(default=False)
 
     objects = ScopedManager(organizer='event__organizer')
 
@@ -2351,8 +2620,7 @@ class InvoiceAddress(models.Model):
     country = FastCountryField(verbose_name=_('Country'), blank=False, blank_label=_('Select country'),
                                countries=CachedCountries)
     state = models.CharField(max_length=255, verbose_name=pgettext_lazy('address', 'State'), blank=True)
-    vat_id = models.CharField(max_length=255, blank=True, verbose_name=_('VAT ID'),
-                              help_text=_('Only for business customers within the EU.'))
+    vat_id = models.CharField(max_length=255, blank=True, verbose_name=_('VAT ID'))
     vat_id_validated = models.BooleanField(default=False)
     custom_field = models.CharField(max_length=255, null=True, blank=True)
     internal_reference = models.TextField(

src/pretix/base/models/organizer.py

@@ -97,10 +97,21 @@ class Organizer(LoggedModel):
         return self.name
 
     def save(self, *args, **kwargs):
+        is_new = not self.pk
         obj = super().save(*args, **kwargs)
-        self.get_cache().clear()
+        if is_new:
+            self.set_defaults()
+        else:
+            self.get_cache().clear()
         return obj
 
+    def set_defaults(self):
+        """
+        This will be called after organizer creation.
+        This way, we can use this to introduce new default settings to pretix that do not affect existing organizers.
+        """
+        self.settings.cookie_consent = True
+
     def get_cache(self):
         """
         Returns an :py:class:`ObjectRelatedCache` object. This behaves equivalent to

src/pretix/base/models/tax.py

@@ -25,7 +25,6 @@ from decimal import Decimal
 from django.core.exceptions import ValidationError
 from django.db import models
 from django.utils.formats import localize
-from django.utils.timezone import get_current_timezone, now
 from django.utils.translation import gettext_lazy as _, pgettext
 from i18nfield.fields import I18nCharField
 from i18nfield.strings import LazyI18nString
@@ -93,7 +92,7 @@ TAXED_ZERO = TaxedPrice(
 
 EU_COUNTRIES = {
     'AT', 'BE', 'BG', 'HR', 'CY', 'CZ', 'DK', 'EE', 'FI', 'FR', 'DE', 'GR', 'HU', 'IE', 'IT', 'LV', 'LT', 'LU', 'MT',
-    'NL', 'PL', 'PT', 'RO', 'SK', 'SI', 'ES', 'SE', 'GB'
+    'NL', 'PL', 'PT', 'RO', 'SK', 'SI', 'ES', 'SE',
 }
 EU_CURRENCIES = {
     'BG': 'BGN',
@@ -106,17 +105,21 @@ EU_CURRENCIES = {
     'RO': 'RON',
     'SE': 'SEK'
 }
+VAT_ID_COUNTRIES = EU_COUNTRIES | {'CH'}
 
 
 def is_eu_country(cc):
     cc = str(cc)
-    if cc == 'GB':
-        return now().astimezone(get_current_timezone()).year <= 2020
-    else:
-        return cc in EU_COUNTRIES
+    return cc in EU_COUNTRIES
+
+
+def ask_for_vat_id(cc):
+    cc = str(cc)
+    return cc in VAT_ID_COUNTRIES
 
 
 def cc_to_vat_prefix(country_code):
+    country_code = str(country_code)
     if country_code == 'GR':
         return 'EL'
     return country_code
@@ -162,10 +165,13 @@ class TaxRule(LoggedModel):
         pass
 
     def allow_delete(self):
-        from pretix.base.models.orders import OrderFee, OrderPosition
+        from pretix.base.models.orders import (
+            OrderFee, OrderPosition, Transaction,
+        )
 
         return (
-            not OrderFee.objects.filter(tax_rule=self, order__event=self.event).exists()
+            not Transaction.objects.filter(tax_rule=self, order__event=self.event).exists()
+            and not OrderFee.objects.filter(tax_rule=self, order__event=self.event).exists()
             and not OrderPosition.all.filter(tax_rule=self, order__event=self.event).exists()
             and not self.event.items.filter(tax_rule=self).exists()
             and self.event.settings.tax_rate_default != self

src/pretix/base/services/checkin.py

@@ -736,7 +736,11 @@ def process_exit_all(sender, **kwargs):
         exit_all_at__isnull=False
     ).select_related('event', 'event__organizer')
     for cl in qs:
-        for p in cl.positions_inside:
+        positions = cl.positions_inside.filter(
+            Q(last_exit__isnull=True) | Q(last_exit__lte=cl.exit_all_at),
+            last_entry__lte=cl.exit_all_at,
+        )
+        for p in positions:
             with scope(organizer=cl.event.organizer):
                 ci = Checkin.objects.create(
                     position=p, list=cl, auto_checked_in=True, type=Checkin.TYPE_EXIT, datetime=cl.exit_all_at
@@ -748,6 +752,9 @@ def process_exit_all(sender, **kwargs):
             cl.event.settings.delete(f'autocheckin_dst_hack_{cl.pk}')
         try:
             cl.exit_all_at = make_aware(datetime.combine(d.date() + timedelta(days=1), d.time()), cl.event.timezone)
+        except pytz.exceptions.AmbiguousTimeError:
+            cl.exit_all_at = make_aware(datetime.combine(d.date() + timedelta(days=1), d.time()), cl.event.timezone,
+                                        is_dst=False)
         except pytz.exceptions.NonExistentTimeError:
             cl.event.settings.set(f'autocheckin_dst_hack_{cl.pk}', True)
             d += timedelta(hours=1)

src/pretix/base/services/invoices.py

@@ -69,6 +69,10 @@ from pretix.helpers.models import modelcopy
 logger = logging.getLogger(__name__)
 
 
+def _location_oneliner(loc):
+    return ', '.join([l.strip() for l in loc.splitlines() if l and l.strip()])
+
+
 @transaction.atomic
 def build_invoice(invoice: Invoice) -> Invoice:
     invoice.locale = invoice.event.settings.get('invoice_language', invoice.event.settings.locale)
@@ -176,19 +180,38 @@ def build_invoice(invoice: Invoice) -> Invoice:
         reverse_charge = False
 
         positions.sort(key=lambda p: p.sort_key)
+        fees = list(invoice.order.fees.all())
+
+        locations = {
+            str((p.subevent or invoice.event).location) if (p.subevent or invoice.event).location else None
+            for p in positions
+        }
+        if fees and invoice.event.has_subevents:
+            locations.add(None)
 
         tax_texts = []
+
+        if invoice.event.settings.invoice_event_location and len(locations) == 1 and list(locations)[0] is not None:
+            tax_texts.append(pgettext("invoice", "Event location: {location}").format(
+                location=_location_oneliner(str(list(locations)[0]))
+            ))
+
         for i, p in enumerate(positions):
             if not invoice.event.settings.invoice_include_free and p.price == Decimal('0.00') and not p.addon_c:
                 continue
 
+            location = str((p.subevent or invoice.event).location) if (p.subevent or invoice.event).location else None
+
             desc = str(p.item.name)
             if p.variation:
                 desc += " - " + str(p.variation.value)
             if p.addon_to_id:
                 desc = "  + " + desc
             if invoice.event.settings.invoice_attendee_name and p.attendee_name:
-                desc += "<br />" + pgettext("invoice", "Attendee: {name}").format(name=p.attendee_name)
+                desc += "<br />" + pgettext("invoice", "Attendee: {name}").format(
+                    name=p.attendee_name
+                )
+
             for recv, resp in invoice_line_text.send(sender=invoice.event, position=p):
                 if resp:
                     desc += "<br/>" + resp
@@ -204,6 +227,12 @@ def build_invoice(invoice: Invoice) -> Invoice:
 
             if invoice.event.has_subevents:
                 desc += "<br />" + pgettext("subevent", "Date: {}").format(p.subevent)
+
+            if invoice.event.settings.invoice_event_location and location and len(locations) > 1:
+                desc += "<br />" + pgettext("invoice", "Event location: {location}").format(
+                    location=_location_oneliner(location)
+                )
+
             InvoiceLine.objects.create(
                 position=i,
                 invoice=invoice,
@@ -216,6 +245,7 @@ def build_invoice(invoice: Invoice) -> Invoice:
                 attendee_name=p.attendee_name if invoice.event.settings.invoice_attendee_name else None,
                 event_date_from=p.subevent.date_from if invoice.event.has_subevents else invoice.event.date_from,
                 event_date_to=p.subevent.date_to if invoice.event.has_subevents else invoice.event.date_to,
+                event_location=location if invoice.event.settings.invoice_event_location else None,
                 tax_rate=p.tax_rate, tax_name=p.tax_rule.name if p.tax_rule else ''
             )
 
@@ -228,7 +258,7 @@ def build_invoice(invoice: Invoice) -> Invoice:
                     tax_texts.append(tax_text)
 
         offset = len(positions)
-        for i, fee in enumerate(invoice.order.fees.all()):
+        for i, fee in enumerate(fees):
             if fee.fee_type == OrderFee.FEE_TYPE_OTHER and fee.description:
                 fee_title = fee.description
             else:
@@ -242,6 +272,12 @@ def build_invoice(invoice: Invoice) -> Invoice:
                 gross_value=fee.value,
                 event_date_from=None if invoice.event.has_subevents else invoice.event.date_from,
                 event_date_to=None if invoice.event.has_subevents else invoice.event.date_to,
+                event_location=(
+                    None if invoice.event.has_subevents
+                    else (str(invoice.event.location)
+                          if invoice.event.settings.invoice_event_location and invoice.event.location
+                          else None)
+                ),
                 tax_value=fee.tax_value,
                 tax_rate=fee.tax_rate,
                 tax_name=fee.tax_rule.name if fee.tax_rule else '',
@@ -291,6 +327,7 @@ def generate_cancellation(invoice: Invoice, trigger_pdf=True):
     cancellation.payment_provider_text = ''
     cancellation.file = None
     cancellation.sent_to_organizer = None
+    cancellation.sent_to_customer = None
     with language(invoice.locale, invoice.event.settings.region):
         cancellation.invoice_from = invoice.event.settings.get('invoice_address_from')
         cancellation.invoice_from_name = invoice.event.settings.get('invoice_address_from_name')
@@ -346,8 +383,8 @@ def invoice_pdf_task(invoice: int):
             i.file.delete()
         with language(i.locale, i.event.settings.region):
             fname, ftype, fcontent = i.event.invoice_renderer.generate(i)
-            i.file.save(fname, ContentFile(fcontent))
-            i.save()
+            i.file.save(fname, ContentFile(fcontent), save=False)
+            i.save(update_fields=['file'])
             return i.file.name
 
 

src/pretix/base/services/mail.py

@@ -32,7 +32,7 @@
 # Unless required by applicable law or agreed to in writing, software distributed under the Apache License 2.0 is
 # distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
 # License for the specific language governing permissions and limitations under the License.
-
+import hashlib
 import inspect
 import logging
 import os
@@ -57,7 +57,7 @@ from django.core.mail import (
 from django.core.mail.message import SafeMIMEText
 from django.db import transaction
 from django.template.loader import get_template
-from django.utils.timezone import override
+from django.utils.timezone import now, override
 from django.utils.translation import gettext as _, pgettext
 from django_scopes import scope, scopes_disabled
 from i18nfield.strings import LazyI18nString
@@ -438,6 +438,7 @@ def mail_send_task(self, *args, to: List[str], subject: str, body: str, html: st
 
             email = email_filter.send_chained(event, 'message', message=email, order=order, user=user)
 
+        invoices_sent = []
         if invoices:
             invoices = Invoice.objects.filter(pk__in=invoices)
             for inv in invoices:
@@ -449,6 +450,7 @@ def mail_send_task(self, *args, to: List[str], subject: str, body: str, html: st
                                 inv.file.file.read(),
                                 'application/pdf'
                             )
+                        invoices_sent.append(inv)
                     except:
                         logger.exception('Could not attach invoice to email')
                         pass
@@ -472,10 +474,30 @@ def mail_send_task(self, *args, to: List[str], subject: str, body: str, html: st
         try:
             backend.send_messages([email])
         except (smtplib.SMTPResponseException, smtplib.SMTPSenderRefused) as e:
-            if e.smtp_code in (101, 111, 421, 422, 431, 442, 447, 452):
-                # Most likely temporary, retry again (but pretty soon)
+            if e.smtp_code in (101, 111, 421, 422, 431, 432, 442, 447, 452):
+                if e.smtp_code == 432 and settings.HAS_REDIS:
+                    # This is likely Microsoft Exchange Online which has a pretty bad rate limit of max. 3 concurrent
+                    # SMTP connections which is *easily* exceeded with many celery threads. Just retrying with exponential
+                    # backoff won't be good enough if we have a lot of emails, instead we'll need to make sure our retry
+                    # intervals scatter such that the email won't all be retried at the same time again and cause the
+                    # same problem.
+                    # See also https://docs.microsoft.com/en-us/exchange/troubleshoot/send-emails/smtp-submission-improvements
+                    from django_redis import get_redis_connection
+
+                    redis_key = "pretix_mail_retry_" + hashlib.sha1(f"{getattr(backend, 'username', '_')}@{getattr(backend, 'host', '_')}".encode()).hexdigest()
+                    rc = get_redis_connection("redis")
+                    cnt = rc.incr(redis_key)
+                    rc.expire(redis_key, 300)
+
+                    max_retries = 10
+                    retry_after = 30 + cnt * 10
+                else:
+                    # Most likely some other kind of temporary failure, retry again (but pretty soon)
+                    max_retries = 5
+                    retry_after = 2 ** (self.request.retries * 3)  # max is 2 ** (4*3) = 4096 seconds = 68 minutes
+
                 try:
-                    self.retry(max_retries=5, countdown=2 ** (self.request.retries * 3))  # max is 2 ** (4*3) = 4096 seconds = 68 minutes
+                    self.retry(max_retries=max_retries, countdown=retry_after)
                 except MaxRetriesExceededError:
                     if log_target:
                         log_target.log_action(
@@ -558,6 +580,10 @@ def mail_send_task(self, *args, to: List[str], subject: str, body: str, html: st
                 )
             logger.exception('Error sending email')
             raise SendMailException('Failed to send an email to {}.'.format(to))
+        else:
+            for i in invoices_sent:
+                i.sent_to_customer = now()
+                i.save(update_fields=['sent_to_customer'])
 
 
 def mail_send(*args, **kwargs):

src/pretix/base/services/orderimport.py

@@ -33,6 +33,7 @@ from pretix.base.models import (
     CachedFile, Event, InvoiceAddress, Order, OrderPayment, OrderPosition,
     User,
 )
+from pretix.base.models.orders import Transaction
 from pretix.base.orderimport import get_all_columns
 from pretix.base.services.invoices import generate_invoice, invoice_qualified
 from pretix.base.services.tasks import ProfiledEventTask
@@ -146,6 +147,7 @@ def import_orders(event: Event, fileid: str, settings: dict, locale: str, user)
         # quota check?
         with event.lock():
             with transaction.atomic():
+                save_transactions = []
                 for o in orders:
                     o.total = sum([c.price for c in o._positions])  # currently no support for fees
                     if o.total == Decimal('0.00'):
@@ -187,6 +189,8 @@ def import_orders(event: Event, fileid: str, settings: dict, locale: str, user)
                         user=user,
                         data={'source': 'import'}
                     )
+                    save_transactions += o.create_transactions(is_new=True, fees=[], positions=o._positions, save=False)
+                Transaction.objects.bulk_create(save_transactions)
 
             for o in orders:
                 with language(o.locale, event.settings.region):

src/pretix/base/services/orders.py

@@ -35,7 +35,7 @@
 
 import json
 import logging
-from collections import Counter, namedtuple
+from collections import Counter, defaultdict, namedtuple
 from datetime import datetime, time, timedelta
 from decimal import Decimal
 from typing import List, Optional
@@ -46,7 +46,7 @@ from django.core.cache import cache
 from django.core.exceptions import ValidationError
 from django.db import transaction
 from django.db.models import (
-    Exists, F, IntegerField, Max, Min, OuterRef, Q, Sum, Value,
+    Count, Exists, F, IntegerField, Max, Min, OuterRef, Q, Sum, Value,
 )
 from django.db.models.functions import Coalesce, Greatest
 from django.db.transaction import get_connection
@@ -73,7 +73,7 @@ from pretix.base.models.orders import (
     InvoiceAddress, OrderFee, OrderRefund, generate_secret,
 )
 from pretix.base.models.organizer import TeamAPIToken
-from pretix.base.models.tax import TaxRule
+from pretix.base.models.tax import TAXED_ZERO, TaxedPrice, TaxRule
 from pretix.base.payment import BasePaymentProvider, PaymentException
 from pretix.base.reldate import RelativeDateWrapper
 from pretix.base.secrets import assign_ticket_secret
@@ -122,8 +122,7 @@ error_messages = {
                          'from your cart.'),
     'voucher_invalid_item': _('The voucher code used for one of the items in your cart is not valid for this item. We '
                               'removed this item from your cart.'),
-    'voucher_required': _('You need a valid voucher code to order one of the products in your cart. We removed this '
-                          'item from your cart.'),
+    'voucher_required': _('You need a valid voucher code to order one of the products.'),
     'some_subevent_not_started': _('The presale period for one of the events in your cart has not yet started. The '
                                    'affected positions have been removed from your cart.'),
     'some_subevent_ended': _('The presale period for one of the events in your cart has ended. The affected '
@@ -131,6 +130,13 @@ error_messages = {
     'seat_invalid': _('One of the seats in your order was invalid, we removed the position from your cart.'),
     'seat_unavailable': _('One of the seats in your order has been taken in the meantime, we removed the position from your cart.'),
     'country_blocked': _('One of the selected products is not available in the selected country.'),
+    'not_for_sale': _('You selected a product which is not available for sale.'),
+    'addon_invalid_base': _('You can not select an add-on for the selected product.'),
+    'addon_duplicate_item': _('You can not select two variations of the same add-on product.'),
+    'addon_max_count': _('You can select at most %(max)s add-ons from the category %(cat)s for the product %(base)s.'),
+    'addon_min_count': _('You need to select at least %(min)s add-ons from the category %(cat)s for the '
+                         'product %(base)s.'),
+    'addon_no_multi': _('You can select every add-ons from the category %(cat)s for the product %(base)s at most once.'),
 }
 
 logger = logging.getLogger(__name__)
@@ -181,6 +187,7 @@ def reactivate_order(order: Order, force: bool=False, user: User=None, auth=None
                     for m in position.granted_memberships.all():
                         m.canceled = False
                         m.save()
+                order.create_transactions()
         else:
             raise OrderError(is_available)
 
@@ -202,6 +209,7 @@ def extend_order(order: Order, new_date: datetime, force: bool=False, user: User
     if new_date < now():
         raise OrderError(_('The new expiry date needs to be in the future.'))
 
+    @transaction.atomic
     def change(was_expired=True):
         order.expires = new_date
         if was_expired:
@@ -221,6 +229,7 @@ def extend_order(order: Order, new_date: datetime, force: bool=False, user: User
             num_invoices = order.invoices.filter(is_cancellation=False).count()
             if num_invoices > 0 and order.invoices.filter(is_cancellation=True).count() >= num_invoices and invoice_qualified(order):
                 generate_invoice(order)
+            order.create_transactions()
 
     if order.status == Order.STATUS_PENDING:
         change(was_expired=False)
@@ -262,6 +271,7 @@ def mark_order_expired(order, user=None, auth=None):
         i = order.invoices.filter(is_cancellation=False).last()
         if i and not i.refered.exists():
             generate_cancellation(i)
+        order.create_transactions()
 
     order_expired.send(order.event, order=order)
     return order
@@ -280,6 +290,7 @@ def approve_order(order, user=None, send_mail: bool=True, auth=None, force=False
         order.require_approval = False
         order.set_expires(now(), order.event.subevents.filter(id__in=[p.subevent_id for p in order.positions.all()]))
         order.save(update_fields=['require_approval', 'expires'])
+        order.create_transactions()
 
         order.log_action('pretix.event.order.approved', user=user, auth=auth)
         if order.total == Decimal('0.00'):
@@ -352,6 +363,7 @@ def deny_order(order, comment='', user=None, send_mail: bool=True, auth=None):
         for position in order.positions.all():
             if position.voucher:
                 Voucher.objects.filter(pk=position.voucher.pk).update(redeemed=Greatest(0, F('redeemed') - 1))
+        order.create_transactions()
 
     order_denied.send(order.event, order=order)
 
@@ -472,6 +484,8 @@ def _cancel_order(order, user=None, send_mail: bool=True, api_token=None, device
                          data={'cancellation_fee': cancellation_fee})
         order.cancellation_requests.all().delete()
 
+        order.create_transactions()
+
         if send_mail:
             email_template = order.event.settings.mail_text_order_canceled
             with language(order.locale, order.event.settings.region):
@@ -904,7 +918,8 @@ def _create_order(event: Event, email: str, positions: List[CartPosition], now_d
                 fee=pf
             )
 
-        OrderPosition.transform_cart_positions(positions, order)
+        orderpositions = OrderPosition.transform_cart_positions(positions, order)
+        order.create_transactions(positions=orderpositions, fees=fees, is_new=True)
         order.log_action('pretix.event.order.placed')
         if order.require_approval:
             order.log_action('pretix.event.order.placed.require_approval')
@@ -1252,15 +1267,15 @@ class OrderChangeManager:
     ItemOperation = namedtuple('ItemOperation', ('position', 'item', 'variation'))
     SubeventOperation = namedtuple('SubeventOperation', ('position', 'subevent'))
     SeatOperation = namedtuple('SubeventOperation', ('position', 'seat'))
-    PriceOperation = namedtuple('PriceOperation', ('position', 'price'))
+    PriceOperation = namedtuple('PriceOperation', ('position', 'price', 'price_diff'))
     TaxRuleOperation = namedtuple('TaxRuleOperation', ('position', 'tax_rule'))
     MembershipOperation = namedtuple('MembershipOperation', ('position', 'membership'))
-    CancelOperation = namedtuple('CancelOperation', ('position',))
+    CancelOperation = namedtuple('CancelOperation', ('position', 'price_diff'))
     AddOperation = namedtuple('AddOperation', ('item', 'variation', 'price', 'addon_to', 'subevent', 'seat', 'membership'))
     SplitOperation = namedtuple('SplitOperation', ('position',))
-    FeeValueOperation = namedtuple('FeeValueOperation', ('fee', 'value'))
-    AddFeeOperation = namedtuple('AddFeeOperation', ('fee',))
-    CancelFeeOperation = namedtuple('CancelFeeOperation', ('fee',))
+    FeeValueOperation = namedtuple('FeeValueOperation', ('fee', 'value', 'price_diff'))
+    AddFeeOperation = namedtuple('AddFeeOperation', ('fee', 'price_diff'))
+    CancelFeeOperation = namedtuple('CancelFeeOperation', ('fee', 'price_diff'))
     RegenerateSecretOperation = namedtuple('RegenerateSecretOperation', ('position',))
 
     def __init__(self, order: Order, user=None, auth=None, notify=True, reissue_invoice=True):
@@ -1377,7 +1392,7 @@ class OrderChangeManager:
         if self.order.event.settings.invoice_include_free or price.gross != Decimal('0.00') or position.price != Decimal('0.00'):
             self._invoice_dirty = True
 
-        self._operations.append(self.PriceOperation(position, price))
+        self._operations.append(self.PriceOperation(position, price, price.gross - position.price))
 
     def change_tax_rule(self, position_or_fee, tax_rule: TaxRule):
         self._operations.append(self.TaxRuleOperation(position_or_fee, tax_rule))
@@ -1417,28 +1432,28 @@ class OrderChangeManager:
                     new_tax = tax_rule.tax(pos.price, base_price_is='gross', currency=self.event.currency,
                                            override_tax_rate=new_rate)
                 self._totaldiff += new_tax.gross - pos.price
-                self._operations.append(self.PriceOperation(pos, new_tax))
+                self._operations.append(self.PriceOperation(pos, new_tax, new_tax.gross - pos.price))
 
     def cancel_fee(self, fee: OrderFee):
         self._totaldiff -= fee.value
-        self._operations.append(self.CancelFeeOperation(fee))
+        self._operations.append(self.CancelFeeOperation(fee, -fee.value))
         self._invoice_dirty = True
 
     def add_fee(self, fee: OrderFee):
         self._totaldiff += fee.value
         self._invoice_dirty = True
-        self._operations.append(self.AddFeeOperation(fee))
+        self._operations.append(self.AddFeeOperation(fee, fee.value))
 
     def change_fee(self, fee: OrderFee, value: Decimal):
         value = (fee.tax_rule or TaxRule.zero()).tax(value, base_price_is='gross')
         self._totaldiff += value.gross - fee.value
         self._invoice_dirty = True
-        self._operations.append(self.FeeValueOperation(fee, value))
+        self._operations.append(self.FeeValueOperation(fee, value, value.gross - fee.value))
 
     def cancel(self, position: OrderPosition):
         self._totaldiff -= position.price
         self._quotadiff.subtract(position.quotas)
-        self._operations.append(self.CancelOperation(position))
+        self._operations.append(self.CancelOperation(position, -position.price))
         if position.seat:
             self._seatdiff.subtract([position.seat])
 
@@ -1463,7 +1478,7 @@ class OrderChangeManager:
         try:
             if price is None:
                 price = get_price(item, variation, subevent=subevent, invoice_address=self._invoice_address)
-            else:
+            elif not isinstance(price, TaxedPrice):
                 price = item.tax(price, base_price_is='gross', invoice_address=self._invoice_address)
         except TaxRule.SaleNotAllowed:
             raise OrderError(self.error_messages['tax_rule_country_blocked'])
@@ -1506,6 +1521,190 @@ class OrderChangeManager:
 
         self._operations.append(self.SplitOperation(position))
 
+    def set_addons(self, addons):
+        if self._operations:
+            raise ValueError("Setting addons should be the first/only operation")
+
+        # Prepare various containers to hold data later
+        current_addons = defaultdict(lambda: defaultdict(list))  # OrderPos -> currently attached add-ons
+        input_addons = defaultdict(Counter)  # OrderPos -> final desired set of add-ons
+        selected_addons = defaultdict(Counter)  # OrderPos, ItemAddOn -> final desired set of add-ons
+        opcache = {}  # OrderPos.pk -> OrderPos
+        quota_diff = Counter()  # Quota -> Number of usages
+        available_categories = defaultdict(set)  # OrderPos -> Category IDs to choose from
+        price_included = defaultdict(dict)  # OrderPos -> CategoryID -> bool(price is included)
+        toplevel_op = self.order.positions.filter(
+            addon_to__isnull=True
+        ).prefetch_related(
+            'addons', 'item__addons', 'item__addons__addon_category'
+        ).select_related('item', 'variation')
+
+        _items_cache = {
+            i.pk: i
+            for i in self.event.items.select_related('category').prefetch_related(
+                'addons', 'bundles', 'addons__addon_category', 'quotas'
+            ).annotate(
+                has_variations=Count('variations'),
+            ).filter(
+                id__in=[a['item'] for a in addons]
+            ).order_by()
+        }
+        _variations_cache = {
+            v.pk: v
+            for v in ItemVariation.objects.filter(item__event=self.event).prefetch_related(
+                'quotas'
+            ).select_related('item', 'item__event').filter(
+                id__in=[a['variation'] for a in addons if a.get('variation')]
+            ).order_by()
+        }
+
+        # Prefill some of the cache containers
+        for op in toplevel_op:
+            if op.canceled:
+                continue
+            available_categories[op.pk] = {iao.addon_category_id for iao in op.item.addons.all()}
+            price_included[op.pk] = {iao.addon_category_id: iao.price_included for iao in op.item.addons.all()}
+            opcache[op.pk] = op
+            for a in op.addons.all():
+                if a.canceled:
+                    continue
+
+                if not a.is_bundled:
+                    current_addons[op][a.item_id, a.variation_id].append(a)
+
+        # Create operations, perform various checks
+        for a in addons:
+            # Check whether the specified items are part of what we just fetched from the database
+            # If they are not, the user supplied item IDs which either do not exist or belong to
+            # a different event
+            if a['item'] not in _items_cache or (a['variation'] and a['variation'] not in _variations_cache):
+                raise OrderError(error_messages['not_for_sale'])
+
+            # Only attach addons to things that are actually in this user's cart
+            if a['addon_to'] not in opcache:
+                raise OrderError(error_messages['addon_invalid_base'])
+
+            op = opcache[a['addon_to']]
+            item = _items_cache[a['item']]
+            variation = _variations_cache[a['variation']] if a['variation'] is not None else None
+
+            if item.category_id not in available_categories[op.pk]:
+                raise OrderError(error_messages['addon_invalid_base'])
+
+            # Fetch all quotas. If there are no quotas, this item is not allowed to be sold.
+            quotas = list(item.quotas.filter(subevent=op.subevent)
+                          if variation is None else variation.quotas.filter(subevent=op.subevent))
+            if not quotas:
+                raise OrderError(error_messages['unavailable'])
+
+            if (a['item'], a['variation']) in input_addons[op.id]:
+                raise OrderError(error_messages['addon_duplicate_item'])
+
+            if item.require_voucher or op.item.hide_without_voucher or (op.variation and op.variation.hide_without_voucher):
+                raise OrderError(error_messages['voucher_required'])
+
+            if not item.is_available() or (variation and not variation.is_available()):
+                raise OrderError(error_messages['unavailable'])
+
+            if self.order.sales_channel not in item.sales_channels or (
+                    variation and self.order.sales_channel not in variation.sales_channels):
+                raise OrderError(error_messages['unavailable'])
+
+            if op.subevent and item.pk in op.subevent.item_overrides and not op.subevent.item_overrides[op.item.pk].is_available():
+                raise OrderError(error_messages['not_for_sale'])
+
+            if op.subevent and variation and variation.pk in op.subevent.var_overrides and \
+                    not op.subevent.var_overrides[variation.pk].is_available():
+                raise OrderError(error_messages['not_for_sale'])
+
+            if item.has_variations and not variation:
+                raise OrderError(error_messages['not_for_sale'])
+
+            if variation and variation.item_id != item.pk:
+                raise OrderError(error_messages['not_for_sale'])
+
+            if op.subevent and op.subevent.presale_start and now() < op.subevent.presale_start:
+                raise OrderError(error_messages['not_started'])
+
+            if (op.subevent and op.subevent.presale_has_ended) or self.event.presale_has_ended:
+                raise OrderError(error_messages['ended'])
+
+            if item.require_bundling:
+                raise OrderError(error_messages['unavailable'])
+
+            input_addons[op.id][a['item'], a['variation']] = a.get('count', 1)
+            selected_addons[op.id, item.category_id][a['item'], a['variation']] = a.get('count', 1)
+
+            if price_included[op.pk].get(item.category_id):
+                price = TAXED_ZERO
+            else:
+                price = get_price(
+                    item, variation, voucher=None, custom_price=a.get('price'), subevent=op.subevent,
+                    custom_price_is_net=self.event.settings.display_net_prices,
+                    invoice_address=self._invoice_address,
+                )
+
+            if a.get('count', 1) > len(current_addons[op][a['item'], a['variation']]):
+                # This add-on is new, add it to the cart
+                for quota in quotas:
+                    quota_diff[quota] += a.get('count', 1) - len(current_addons[op][a['item'], a['variation']])
+
+                for i in range(a.get('count', 1) - len(current_addons[op][a['item'], a['variation']])):
+                    self.add_position(
+                        item=item, variation=variation, price=price,
+                        addon_to=op, subevent=op.subevent, seat=None,
+                    )
+
+        # Check constraints on the add-on combinations
+        for op in toplevel_op:
+            item = op.item
+            for iao in item.addons.all():
+                selected = selected_addons[op.id, iao.addon_category_id]
+                n_per_i = Counter()
+                for (i, v), c in selected.items():
+                    n_per_i[i] += c
+                if sum(selected.values()) > iao.max_count:
+                    # TODO: Proper i18n
+                    # TODO: Proper pluralization
+                    raise OrderError(
+                        error_messages['addon_max_count'],
+                        {
+                            'base': str(item.name),
+                            'max': iao.max_count,
+                            'cat': str(iao.addon_category.name),
+                        }
+                    )
+                elif sum(selected.values()) < iao.min_count:
+                    # TODO: Proper i18n
+                    # TODO: Proper pluralization
+                    raise OrderError(
+                        error_messages['addon_min_count'],
+                        {
+                            'base': str(item.name),
+                            'min': iao.min_count,
+                            'cat': str(iao.addon_category.name),
+                        }
+                    )
+                elif any(v > 1 for v in n_per_i.values()) and not iao.multi_allowed:
+                    raise OrderError(
+                        error_messages['addon_no_multi'],
+                        {
+                            'base': str(item.name),
+                            'cat': str(iao.addon_category.name),
+                        }
+                    )
+
+        # Detect removed add-ons and create RemoveOperations
+        for cp, al in list(current_addons.items()):
+            for k, v in al.items():
+                input_num = input_addons[cp.id].get(k, 0)
+                current_num = len(current_addons[cp].get(k, []))
+                if input_num < current_num:
+                    for a in current_addons[cp][k][:current_num - input_num]:
+                        if a.canceled:
+                            continue
+                        self.cancel(a)
+
     def _check_seats(self):
         for seat, diff in self._seatdiff.items():
             if diff <= 0:
@@ -1552,17 +1751,16 @@ class OrderChangeManager:
                 self.order.save()
             elif self.open_payment:
                 try:
-                    with transaction.atomic():
-                        self.open_payment.payment_provider.cancel_payment(self.open_payment)
-                        self.order.log_action(
-                            'pretix.event.order.payment.canceled',
-                            {
-                                'local_id': self.open_payment.local_id,
-                                'provider': self.open_payment.provider,
-                            },
-                            user=self.user,
-                            auth=self.auth
-                        )
+                    self.open_payment.payment_provider.cancel_payment(self.open_payment)
+                    self.order.log_action(
+                        'pretix.event.order.payment.canceled',
+                        {
+                            'local_id': self.open_payment.local_id,
+                            'provider': self.open_payment.provider,
+                        },
+                        user=self.user,
+                        auth=self.auth
+                    )
                 except PaymentException as e:
                     self.order.log_action(
                         'pretix.event.order.payment.canceled.failed',
@@ -1577,12 +1775,11 @@ class OrderChangeManager:
         elif self.order.status in (Order.STATUS_PENDING, Order.STATUS_EXPIRED) and self._totaldiff > 0:
             if self.open_payment:
                 try:
-                    with transaction.atomic():
-                        self.open_payment.payment_provider.cancel_payment(self.open_payment)
-                        self.order.log_action('pretix.event.order.payment.canceled', {
-                            'local_id': self.open_payment.local_id,
-                            'provider': self.open_payment.provider,
-                        }, user=self.user, auth=self.auth)
+                    self.open_payment.payment_provider.cancel_payment(self.open_payment)
+                    self.order.log_action('pretix.event.order.payment.canceled', {
+                        'local_id': self.open_payment.local_id,
+                        'provider': self.open_payment.provider,
+                    }, user=self.user, auth=self.auth)
                 except PaymentException as e:
                     self.order.log_action(
                         'pretix.event.order.payment.canceled.failed',
@@ -2122,10 +2319,13 @@ class OrderChangeManager:
             self._recalculate_total_and_payment_fee()
             if self.order.status in (Order.STATUS_PENDING, Order.STATUS_PAID):
                 self._reissue_invoice()
+            self._check_paid_price_change()
+            self._check_paid_to_free()
             self._clear_tickets_cache()
             self.order.touch()
-        self._check_paid_price_change()
-        self._check_paid_to_free()
+            self.order.create_transactions()
+            if self.split_order:
+                self.split_order.create_transactions()
 
         if self.notify:
             notify_user_changed_order(
@@ -2399,6 +2599,7 @@ def change_payment_provider(order: Order, payment_provider, amount=None, new_pay
             generate_cancellation(i)
             generate_invoice(order)
 
+    order.create_transactions()
     return old_fee, new_fee, fee, new_payment
 
 

src/pretix/base/services/tax.py

@@ -0,0 +1,134 @@
+#
+# This file is part of pretix (Community Edition).
+#
+# Copyright (C) 2014-2020 Raphael Michel and contributors
+# Copyright (C) 2020-2021 rami.io GmbH and contributors
+#
+# This program is free software: you can redistribute it and/or modify it under the terms of the GNU Affero General
+# Public License as published by the Free Software Foundation in version 3 of the License.
+#
+# ADDITIONAL TERMS APPLY: Pursuant to Section 7 of the GNU Affero General Public License, additional terms are
+# applicable granting you additional permissions and placing additional restrictions on your usage of this software.
+# Please refer to the pretix LICENSE file to obtain the full terms applicable to this work. If you did not receive
+# this file, see <https://pretix.eu/about/en/license>.
+#
+# This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied
+# warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU Affero General Public License for more
+# details.
+#
+# You should have received a copy of the GNU Affero General Public License along with this program.  If not, see
+# <https://www.gnu.org/licenses/>.
+#
+import logging
+import os
+import re
+from urllib.error import HTTPError
+
+import vat_moss.errors
+import vat_moss.id
+from django.conf import settings
+from django.utils.translation import gettext_lazy as _
+from zeep import Client, Transport
+from zeep.cache import SqliteCache
+from zeep.exceptions import Fault
+
+from pretix.base.models.tax import cc_to_vat_prefix, is_eu_country
+
+logger = logging.getLogger(__name__)
+
+
+class VATIDError(Exception):
+    def __init__(self, message):
+        self.message = message
+
+
+class VATIDFinalError(VATIDError):
+    pass
+
+
+class VATIDTemporaryError(VATIDError):
+    pass
+
+
+def _validate_vat_id_EU(vat_id, country_code):
+    if vat_id[:2] != cc_to_vat_prefix(country_code):
+        raise VATIDFinalError(_('Your VAT ID does not match the selected country.'))
+
+    try:
+        result = vat_moss.id.validate(vat_id)
+        if result:
+            country_code, normalized_id, company_name = result
+            return normalized_id
+    except (vat_moss.errors.InvalidError, ValueError):
+        raise VATIDFinalError(_('This VAT ID is not valid. Please re-check your input.'))
+    except vat_moss.errors.WebServiceUnavailableError:
+        logger.exception('VAT ID checking failed for country {}'.format(country_code))
+        raise VATIDTemporaryError(_(
+            'Your VAT ID could not be checked, as the VAT checking service of '
+            'your country is currently not available. We will therefore '
+            'need to charge VAT on your invoice. You can get the tax amount '
+            'back via the VAT reimbursement process.'
+        ))
+    except (vat_moss.errors.WebServiceError, HTTPError):
+        logger.exception('VAT ID checking failed for country {}'.format(country_code))
+        raise VATIDTemporaryError(_(
+            'Your VAT ID could not be checked, as the VAT checking service of '
+            'your country returned an incorrect result. We will therefore '
+            'need to charge VAT on your invoice. Please contact support to '
+            'resolve this manually.'
+        ))
+
+
+def _validate_vat_id_CH(vat_id, country_code):
+    if vat_id[:3] != 'CHE':
+        raise VATIDFinalError(_('Your VAT ID does not match the selected country.'))
+
+    vat_id = re.sub('[^A-Z0-9]', '', vat_id.replace('HR', '').replace('MWST', ''))
+    try:
+        transport = Transport(cache=SqliteCache(os.path.join(settings.CACHE_DIR, "validate_vat_id_ch_zeep_cache.db")))
+        client = Client(
+            'https://www.uid-wse.admin.ch/V5.0/PublicServices.svc?wsdl',
+            transport=transport
+        )
+        result = client.service.ValidateUID(uid=vat_id)
+    except Fault as e:
+        if e.message == 'Data_validation_failed':
+            raise VATIDFinalError(_('This VAT ID is not valid. Please re-check your input.'))
+        elif e.message == 'Request_limit_exceeded':
+            logger.exception('VAT ID checking failed for country {} due to request limit'.format(country_code))
+            raise VATIDTemporaryError(_(
+                'Your VAT ID could not be checked, as the VAT checking service of '
+                'your country returned an incorrect result. We will therefore '
+                'need to charge VAT on your invoice. Please contact support to '
+                'resolve this manually.'
+            ))
+        else:
+            logger.exception('VAT ID checking failed for country {}'.format(country_code))
+            raise VATIDTemporaryError(_(
+                'Your VAT ID could not be checked, as the VAT checking service of '
+                'your country returned an incorrect result. We will therefore '
+                'need to charge VAT on your invoice. Please contact support to '
+                'resolve this manually.'
+            ))
+    except:
+        logger.exception('VAT ID checking failed for country {}'.format(country_code))
+        raise VATIDTemporaryError(_(
+            'Your VAT ID could not be checked, as the VAT checking service of '
+            'your country is currently not available. We will therefore '
+            'need to charge VAT on your invoice. You can get the tax amount '
+            'back via the VAT reimbursement process.'
+        ))
+    else:
+        if not result:
+            raise VATIDFinalError(_('This VAT ID is not valid. Please re-check your input.'))
+        return vat_id
+
+
+def validate_vat_id(vat_id, country_code):
+    country_code = str(country_code)
+    if is_eu_country(country_code):
+        return _validate_vat_id_EU(vat_id, country_code)
+    elif country_code == 'CH':
+        return _validate_vat_id_CH(vat_id, country_code)
+
+    raise VATIDTemporaryError(f'VAT ID should not be entered for country {country_code}')

src/pretix/base/settings.py

@@ -48,10 +48,12 @@ from django.core.validators import (
     MaxValueValidator, MinValueValidator, RegexValidator,
 )
 from django.db.models import Model
+from django.utils.functional import lazy
 from django.utils.text import format_lazy
 from django.utils.translation import (
-    gettext_lazy as _, gettext_noop, pgettext, pgettext_lazy,
+    gettext, gettext_lazy as _, gettext_noop, pgettext, pgettext_lazy,
 )
+from django_countries.fields import Country
 from hierarkey.models import GlobalSettingsBase, Hierarkey
 from i18nfield.forms import I18nFormField, I18nTextarea, I18nTextInput
 from i18nfield.strings import LazyI18nString
@@ -61,7 +63,7 @@ from pretix.api.serializers.fields import (
     ListMultipleChoiceField, UploadedFileField,
 )
 from pretix.api.serializers.i18n import I18nField
-from pretix.base.models.tax import TaxRule
+from pretix.base.models.tax import VAT_ID_COUNTRIES, TaxRule
 from pretix.base.reldate import (
     RelativeDateField, RelativeDateTimeField, RelativeDateWrapper,
     SerializerRelativeDateField, SerializerRelativeDateTimeField,
@@ -308,6 +310,17 @@ DEFAULTS = {
             label=_("Show attendee names on invoices"),
         )
     },
+    'invoice_event_location': {
+        'default': 'False',
+        'type': bool,
+        'form_class': forms.BooleanField,
+        'serializer_class': serializers.BooleanField,
+        'form_kwargs': dict(
+            label=_("Show event location on invoices"),
+            help_text=_("The event location will be shown below the list of products if it is the same for all "
+                        "lines. It will be shown on every line if there are different locations.")
+        )
+    },
     'invoice_eu_currencies': {
         'default': 'True',
         'type': bool,
@@ -370,7 +383,11 @@ DEFAULTS = {
         'serializer_class': serializers.BooleanField,
         'form_kwargs': dict(
             label=_("Ask for VAT ID"),
-            help_text=_("Does only work if an invoice address is asked for. VAT ID is not required."),
+            help_text=format_lazy(
+                _("Only works if an invoice address is asked for. VAT ID is never required and only requested from "
+                  "business customers in the following countries: {countries}"),
+                countries=lazy(lambda *args: ', '.join(sorted(gettext(Country(cc).name) for cc in VAT_ID_COUNTRIES)), str)()
+            ),
             widget=forms.CheckboxInput(attrs={'data-checkbox-dependency': '#id_invoice_address_asked'}),
         )
     },
@@ -409,7 +426,7 @@ DEFAULTS = {
         )
     },
     'invoice_include_expire_date': {
-        'default': 'False',
+        'default': 'False',  # default for new events is True
         'type': bool,
         'form_class': forms.BooleanField,
         'serializer_class': serializers.BooleanField,
@@ -465,7 +482,7 @@ DEFAULTS = {
         )
     },
     'invoice_renderer': {
-        'default': 'classic',
+        'default': 'classic',  # default for new events is 'modern1'
         'type': str,
     },
     'ticket_secret_generator': {
@@ -891,7 +908,7 @@ DEFAULTS = {
         'type': str
     },
     'invoice_email_attachment': {
-        'default': 'False',
+        'default': 'False',  # default for new events is True
         'type': bool,
         'form_class': forms.BooleanField,
         'serializer_class': serializers.BooleanField,
@@ -1224,7 +1241,7 @@ DEFAULTS = {
         )
     },
     'event_list_type': {
-        'default': 'list',
+        'default': 'list',  # default for new events is 'calendar'
         'type': str,
         'form_class': forms.ChoiceField,
         'serializer_class': serializers.ChoiceField,
@@ -1284,6 +1301,15 @@ DEFAULTS = {
             label=_("Customers can change the variation of the products they purchased"),
         )
     },
+    'change_allow_user_addons': {
+        'default': 'False',
+        'type': bool,
+        'form_class': forms.BooleanField,
+        'serializer_class': serializers.BooleanField,
+        'form_kwargs': dict(
+            label=_("Customers can change their selected add-on products"),
+        )
+    },
     'change_allow_user_price': {
         'default': 'gte',
         'type': str,
@@ -1486,6 +1512,17 @@ DEFAULTS = {
         ),
         'serializer_class': serializers.URLField,
     },
+    'privacy_url': {
+        'default': None,
+        'type': str,
+        'form_class': forms.URLField,
+        'form_kwargs': dict(
+            label=_("Privacy Policy URL"),
+            help_text=_("This should point e.g. to a part of your website that explains how you use data gathered in "
+                        "your ticket shop."),
+        ),
+        'serializer_class': serializers.URLField,
+    },
     'confirm_texts': {
         'default': LazyI18nStringList(),
         'type': LazyI18nStringList,
@@ -1709,6 +1746,17 @@ Best regards,
 Your {event} team"""))
     },
     'mail_days_order_expire_warning': {
+        'form_class': forms.IntegerField,
+        'serializer_class': serializers.IntegerField,
+        'serializer_kwargs': dict(
+            min_value=0,
+        ),
+        'form_kwargs': dict(
+            label=_("Number of days"),
+            min_value=0,
+            help_text=_("This email will be sent out this many days before the order expires. If the "
+                        "value is 0, the mail will never be sent.")
+        ),
         'type': int,
         'default': '3'
     },
@@ -1957,7 +2005,7 @@ Your {organizer} team"""))
         ),
     },
     'theme_color_success': {
-        'default': '#50A167',
+        'default': '#50a167',
         'type': str,
         'form_class': forms.CharField,
         'serializer_class': serializers.CharField,
@@ -1979,7 +2027,7 @@ Your {organizer} team"""))
         ),
     },
     'theme_color_danger': {
-        'default': '#C44F4F',
+        'default': '#c44f4f',
         'type': str,
         'form_class': forms.CharField,
         'serializer_class': serializers.CharField,
@@ -2001,7 +2049,7 @@ Your {organizer} team"""))
         ),
     },
     'theme_color_background': {
-        'default': '#FFFFFF',
+        'default': '#f5f5f5',
         'type': str,
         'form_class': forms.CharField,
         'serializer_class': serializers.CharField,
@@ -2427,7 +2475,7 @@ Your {organizer} team"""))
         )
     },
     'name_scheme': {
-        'default': 'full',
+        'default': 'full',  # default for new events is 'given_family'
         'type': str
     },
     'giftcard_length': {
@@ -2452,6 +2500,77 @@ Your {organizer} team"""))
                         'many years. If you keep it empty, gift cards do not have an explicit expiry date.'),
         )
     },
+    'cookie_consent': {
+        'default': 'False',
+        'form_class': forms.BooleanField,
+        'serializer_class': serializers.BooleanField,
+        'form_kwargs': dict(
+            label=_("Enable cookie consent management features"),
+        ),
+        'type': bool,
+    },
+    'cookie_consent_dialog_text': {
+        'default': LazyI18nString.from_gettext(gettext_noop(
+            'By clicking "Accept all cookies", you agree to the storing of cookies and use of similar technologies on '
+            'your device.'
+        )),
+        'type': LazyI18nString,
+        'serializer_class': I18nField,
+        'form_class': I18nFormField,
+        'form_kwargs': dict(
+            label=_("Dialog text"),
+            widget=I18nTextarea,
+            widget_kwargs={'attrs': {'rows': '3', 'data-display-dependency': '#id_settings-cookie_consent'}},
+        )
+    },
+    'cookie_consent_dialog_text_secondary': {
+        'default': LazyI18nString.from_gettext(gettext_noop(
+            'We use cookies and similar technologies to gather data that allows us to improve this website and our '
+            'offerings. If you do not agree, we will only use cookies if they are essential to providing the services '
+            'this website offers.'
+        )),
+        'type': LazyI18nString,
+        'serializer_class': I18nField,
+        'form_class': I18nFormField,
+        'form_kwargs': dict(
+            label=_("Secondary dialog text"),
+            widget=I18nTextarea,
+            widget_kwargs={'attrs': {'rows': '3', 'data-display-dependency': '#id_settings-cookie_consent'}},
+        )
+    },
+    'cookie_consent_dialog_title': {
+        'default': LazyI18nString.from_gettext(gettext_noop('Privacy settings')),
+        'type': LazyI18nString,
+        'serializer_class': I18nField,
+        'form_class': I18nFormField,
+        'form_kwargs': dict(
+            label=_('Dialog title'),
+            widget=I18nTextInput,
+            widget_kwargs={'attrs': {'data-display-dependency': '#id_settings-cookie_consent'}},
+        )
+    },
+    'cookie_consent_dialog_button_yes': {
+        'default': LazyI18nString.from_gettext(gettext_noop('Accept all cookies')),
+        'type': LazyI18nString,
+        'serializer_class': I18nField,
+        'form_class': I18nFormField,
+        'form_kwargs': dict(
+            label=_('"Accept" button description'),
+            widget=I18nTextInput,
+            widget_kwargs={'attrs': {'data-display-dependency': '#id_settings-cookie_consent'}},
+        )
+    },
+    'cookie_consent_dialog_button_no': {
+        'default': LazyI18nString.from_gettext(gettext_noop('Required cookies only')),
+        'type': LazyI18nString,
+        'serializer_class': I18nField,
+        'form_class': I18nFormField,
+        'form_kwargs': dict(
+            label=_('"Reject" button description'),
+            widget=I18nTextInput,
+            widget_kwargs={'attrs': {'data-display-dependency': '#id_settings-cookie_consent'}},
+        )
+    },
     'seating_choice': {
         'default': 'True',
         'form_class': forms.BooleanField,

src/pretix/base/templates/pretixbase/email/simple_logo.html

@@ -14,16 +14,17 @@
 </o:OfficeDocumentSettings>
 </xml><![endif]-->
     <style type="text/css">
-        body {
+        body, .container {
             background-color: #eee;
-            background-position: top;
-            background-repeat: repeat-x;
             font-family: "Open Sans", "OpenSans", "Helvetica Neue", Helvetica, Arial, sans-serif;
             font-size: 16px;
             line-height: 1.4;
             color: #333;
             margin: 0;
-            padding-top: 20px;
+            padding: 0;
+        }
+        .container {
+            padding: 20px;
         }
 
         table.layout > tr > td,
@@ -36,7 +37,8 @@
         table.layout > tr > td.logo,
         table.layout > tbody > tr > td.logo,
         table.layout > thead > tr > td.logo {
-            padding: 20px 0 0 0;
+            padding: {% if event.settings.logo_image_large %}0 0 0 0{% else %}20px 0 0 0{% endif %};
+            mso-line-height-rule: at-least;
         }
 
         table.layout > tr > td.header,
@@ -112,10 +114,6 @@
             font-size: 12px;
         }
 
-        .content {
-            padding: 0 18px;
-        }
-
         ::selection {
             background: {{ color }};
             color: #FFF;
@@ -134,6 +132,10 @@
             height: auto;
         }
 
+        img {
+            display: block;
+        }
+
         .content table {
             width: 100%;
         }
@@ -145,7 +147,7 @@
         table.layout > tr > td.containertd,
         table.layout > tbody > tr > td.containertd,
         table.layout > thead > tr > td.containertd {
-            padding: 15px 0;
+            padding: 20px;
         }
 
         a.button {
@@ -163,7 +165,8 @@
         }
 
         .order-button {
-            padding-top: 5px
+            padding-top: 5px;
+            text-align: center;
         }
         .order-button a.button {
             font-size: 12px;
@@ -210,14 +213,14 @@
     <![endif]-->
 </head>
 <body align="center">
+    <table width="100%"><tr><td align="center" class="container">
     <!--[if gte mso 9]>
-    <table width="100%"><tr><td align="center">
-    <table width="600"><tr><td align="center"
+    <table width="600"><tr><td align="center">
     <![endif]-->
     <table class="layout" style="max-width:600px" border="0" cellspacing="0">
         {% if event.settings.logo_image %}
             <tr>
-                <td style="line-height: 0; {% if event.settings.logo_image_large %}padding: 0;{% endif %}" align="center" class="logo">
+                <td align="center" class="logo">
                     {% if event.settings.logo_image_large %}
                         <img src="{% if event.settings.logo_image|thumb:'600_x5000'|first == '/' %}{{ site_url }}{% endif %}{{ event.settings.logo_image|thumb:'600_x5000' }}" alt="{{ event.name }}" style="width:100%" />
                     {% else %}
@@ -228,9 +231,6 @@
         {% endif %}
         <tr>
             <td class="header" align="center">
-                <!--[if gte mso 9]>
-                    <table cellpadding="20"><tr><td align="center">
-                <![endif]-->
                 {% if event %}
                     <h2><a href="{% abseventurl event "presale:event.index" %}" target="_blank">{{ event.name }}</a>
                     </h2>
@@ -243,51 +243,30 @@
                 {% block header %}
                     <h1>{{ subject }}</h1>
                 {% endblock %}
-                <!--[if gte mso 9]>
-                    </td></tr></table>
-                <![endif]-->
             </td>
         </tr>
         <tr>
             <td class="containertd">
-                <!--[if gte mso 9]>
-                        <table cellpadding="20"><tr><td>
-                    <![endif]-->
                 <div class="content">
                     {{ body|safe }}
                 </div>
-                <!--[if gte mso 9]>
-                        </td></tr></table>
-                    <![endif]-->
             </td>
         </tr>
         {% if order %}
             <tr>
                 <td class="order containertd">
-                    <!--[if gte mso 9]>
-                        <table cellpadding="20"><tr><td>
-                    <![endif]-->
                     <div class="content">
                         {% include "pretixbase/email/order_details.html" %}
                     </div>
-                    <!--[if gte mso 9]>
-                        </td></tr></table>
-                    <![endif]-->
                 </td>
             </tr>
         {% endif %}
         {% if signature %}
             <tr>
                 <td class="order containertd">
-                    <!--[if gte mso 9]>
-                        <table cellpadding="20"><tr><td>
-                    <![endif]-->
                     <div class="content">
                         {{ signature | safe }}
                     </div>
-                    <!--[if gte mso 9]>
-                        </td></tr></table>
-                    <![endif]-->
                 </td>
             </tr>
         {% endif %}
@@ -299,7 +278,7 @@
     <br/>
     <!--[if gte mso 9]>
     </td></tr></table>
-    </td></tr></table>
     <![endif]-->
+    </td></tr></table>
 </body>
 </html>

src/pretix/base/templatetags/classname.py

@@ -0,0 +1,29 @@
+#
+# This file is part of pretix (Community Edition).
+#
+# Copyright (C) 2014-2020 Raphael Michel and contributors
+# Copyright (C) 2020-2021 rami.io GmbH and contributors
+#
+# This program is free software: you can redistribute it and/or modify it under the terms of the GNU Affero General
+# Public License as published by the Free Software Foundation in version 3 of the License.
+#
+# ADDITIONAL TERMS APPLY: Pursuant to Section 7 of the GNU Affero General Public License, additional terms are
+# applicable granting you additional permissions and placing additional restrictions on your usage of this software.
+# Please refer to the pretix LICENSE file to obtain the full terms applicable to this work. If you did not receive
+# this file, see <https://pretix.eu/about/en/license>.
+#
+# This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied
+# warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU Affero General Public License for more
+# details.
+#
+# You should have received a copy of the GNU Affero General Public License along with this program.  If not, see
+# <https://www.gnu.org/licenses/>.
+#
+from django import template
+
+register = template.Library()
+
+
+@register.filter
+def classname(obj):
+    return obj.__class__.__name__

src/pretix/base/templatetags/money.py

@@ -34,6 +34,8 @@ register = template.Library()
 def money_filter(value: Decimal, arg='', hide_currency=False):
     if isinstance(value, (float, int)):
         value = Decimal(value)
+    if value is None:
+        value = Decimal('0.00')
     if not isinstance(value, Decimal):
         if value == '':
             return value

src/pretix/base/views/cachedfiles.py

@@ -47,7 +47,7 @@ class DownloadView(TemplateView):
             return HttpResponse('1' if self.object.file else '0')
         elif self.object.file:
             resp = ChunkBasedFileResponse(self.object.file.file, content_type=self.object.type)
-            resp['Content-Disposition'] = 'attachment; filename="{}"'.format(self.object.filename)
+            resp['Content-Disposition'] = 'attachment; filename="{}"'.format(self.object.filename).encode('ascii', 'ignore')
             return resp
         else:
             return super().get(request, *args, **kwargs)

src/pretix/control/forms/event.py

@@ -639,6 +639,7 @@ class CancelSettingsForm(SettingsForm):
         'change_allow_user_variation',
         'change_allow_user_price',
         'change_allow_user_until',
+        'change_allow_user_addons',
     ]
 
     def __init__(self, *args, **kwargs):
@@ -751,6 +752,7 @@ class InvoiceSettingsForm(SettingsForm):
         'invoice_reissue_after_modify',
         'invoice_generate',
         'invoice_attendee_name',
+        'invoice_event_location',
         'invoice_include_expire_date',
         'invoice_numbers_consecutive',
         'invoice_numbers_prefix',

src/pretix/control/forms/item.py

@@ -607,6 +607,7 @@ class ItemUpdateForm(I18nModelForm):
             'issue_giftcard',
             'require_membership',
             'require_membership_types',
+            'require_membership_hidden',
             'grant_membership_type',
             'grant_membership_duration_like_event',
             'grant_membership_duration_days',
@@ -713,6 +714,7 @@ class ItemVariationForm(I18nModelForm):
             'original_price',
             'description',
             'require_membership',
+            'require_membership_hidden',
             'require_membership_types',
             'available_from',
             'available_until',

src/pretix/control/forms/orders.py

@@ -57,7 +57,8 @@ from pretix.base.forms.widgets import (
     DatePickerWidget, SplitDateTimePickerWidget,
 )
 from pretix.base.models import (
-    InvoiceAddress, ItemAddOn, Order, OrderFee, OrderPosition, TaxRule,
+    Invoice, InvoiceAddress, ItemAddOn, Order, OrderFee, OrderPosition,
+    TaxRule,
 )
 from pretix.base.models.event import SubEvent
 from pretix.base.services.pricing import get_price
@@ -609,6 +610,17 @@ class OrderMailForm(forms.Form):
         label=_("Subject"),
         required=True
     )
+    attach_tickets = forms.BooleanField(
+        label=_("Attach tickets"),
+        help_text=_("Will be ignored if all tickets in this order exceed a given size limit to ensure email deliverability."),
+        required=False
+    )
+    attach_invoices = forms.ModelMultipleChoiceField(
+        label=_("Attach invoices"),
+        widget=forms.CheckboxSelectMultiple,
+        required=False,
+        queryset=Invoice.objects.none()
+    )
 
     def _set_field_placeholders(self, fn, base_parameters):
         phs = [
@@ -641,6 +653,7 @@ class OrderMailForm(forms.Form):
             widget=forms.Textarea,
             initial=order.event.settings.mail_text_order_custom_mail.localize(order.locale),
         )
+        self.fields['attach_invoices'].queryset = order.invoices.all()
         self._set_field_placeholders('message', ['event', 'order'])
 
 
@@ -648,6 +661,7 @@ class OrderPositionMailForm(OrderMailForm):
     def __init__(self, *args, **kwargs):
         position = self.position = kwargs.pop('position')
         super().__init__(*args, **kwargs)
+        del self.fields['attach_invoices']
         self.fields['sendto'].initial = position.attendee_email
         self.fields['message'] = forms.CharField(
             label=_("Message"),

src/pretix/control/forms/organizer.py

@@ -307,8 +307,14 @@ class OrganizerSettingsForm(SettingsForm):
         'theme_color_danger',
         'theme_color_background',
         'theme_round_borders',
-        'primary_font'
-
+        'primary_font',
+        'privacy_url',
+        'cookie_consent',
+        'cookie_consent_dialog_title',
+        'cookie_consent_dialog_text',
+        'cookie_consent_dialog_text_secondary',
+        'cookie_consent_dialog_button_yes',
+        'cookie_consent_dialog_button_no',
     ]
 
     organizer_logo_image = ExtFileField(

src/pretix/control/forms/users.py

@@ -70,6 +70,7 @@ class UserEditForm(forms.ModelForm):
             'require_2fa',
             'is_active',
             'is_staff',
+            'needs_password_change',
             'last_login'
         ]
 

src/pretix/control/middleware.py

@@ -69,6 +69,11 @@ class PermissionMiddleware:
         "user.settings.notifications.off",
     )
 
+    EXCEPTIONS_FORCED_PW_CHANGE = (
+        "user.settings",
+        "auth.logout"
+    )
+
     EXCEPTIONS_2FA = (
         "user.settings.2fa",
         "user.settings.2fa.add",
@@ -130,6 +135,9 @@ class PermissionMiddleware:
             if url_name not in ('user.reauth', 'auth.logout'):
                 return redirect(reverse('control:user.reauth') + '?next=' + quote(request.get_full_path()))
 
+        if request.user.needs_password_change and url_name not in self.EXCEPTIONS_FORCED_PW_CHANGE:
+            return redirect(reverse('control:user.settings') + '?next=' + quote(request.get_full_path()))
+
         if not request.user.require_2fa and settings.PRETIX_OBLIGATORY_2FA \
                 and url_name not in self.EXCEPTIONS_2FA:
             return redirect(reverse('control:user.settings.2fa'))

src/pretix/control/templates/pretixcontrol/base.html

@@ -59,7 +59,7 @@
             <script type="text/javascript" src="{% static "colorpicker/bootstrap-colorpicker.js" %}"></script>
             <script type="text/javascript" src="{% static "fileupload/jquery.ui.widget.js" %}"></script>
             <script type="text/javascript" src="{% static "fileupload/jquery.fileupload.js" %}"></script>
-            <script type="text/javascript" src="{% static "lightbox/js/lightbox.min.js" %}"></script>
+            <script type="text/javascript" src="{% static "lightbox/js/lightbox.js" %}"></script>
             <script type="text/javascript" src="{% static "are-you-sure/jquery.are-you-sure.js" %}"></script>
 		{% endcompress %}
         {{ html_head|safe }}
@@ -429,6 +429,15 @@
                         </div>
                     {% endif %}
 
+                    {% if request.user.needs_password_change %}
+                        <div class="alert alert-warning">
+                            {% blocktrans trimmed %}
+                                For security reasons, please change your password before you continue. Afterwards you
+                                will be redirected to your original destination.
+                            {% endblocktrans %}
+                        </div>
+                    {% endif %}
+
                     {% block content %}
                     {% endblock %}
                     <footer>

src/pretix/control/templates/pretixcontrol/event/cancel.html

@@ -42,13 +42,37 @@
             <fieldset>
                 <legend>{% trans "Order changes" %}</legend>
                 <div class="alert alert-info">
-                    {% blocktrans trimmed %}
-                        Allowing users to change their order is a feature under development. Therefore, currently only specific changes (such as changing the variation of a product) are possible. More options might be added later.
-                    {% endblocktrans %}
+                    <p>
+                        {% blocktrans trimmed %}
+                            Allowing customers to change their own orders is a complex process due to the many different options pretix provides. Therefore, this feature currently has the following
+                            limitations:
+                        {% endblocktrans %}
+                    </p>
+                    <ul>
+                        <li>{% trans "It is possible to switch to a different variation of the same product, but not to an entirely different product (except for add-on products)." %}</li>
+                        <li>{% trans "Changing the seat or the event date in an event series will become available in the future, but is not possible now." %}</li>
+                        <li>{% trans "If a change leads to a price change, there will not be a change to fees such as payment, service, or shipping fees, even though an additional payment might be required." %}</li>
+                        <li>{% trans "If an add-on product is newly added, the system currently does not validate if there are required questions or fields that need to be filled out." %}</li>
+                        <li>{% trans "Customers currently cannot switch to a product variation or add an add-on product that requires them to use a voucher or membership." %}</li>
+                        <li>{% trans "Additional constraints and validation steps added by plugins are not enforced." %}</li>
+                    </ul>
                 </div>
                 {% bootstrap_field form.change_allow_user_variation layout="control" %}
-                {% bootstrap_field form.change_allow_user_price layout="control" %}
+                {% bootstrap_field form.change_allow_user_addons layout="control" %}
                 {% bootstrap_field form.change_allow_user_until layout="control" %}
+                {% bootstrap_field form.change_allow_user_price layout="control" %}
+                <div class="alert alert-info">
+                    <p>
+                        {% blocktrans trimmed %}
+                            If the change leads to a price reduction and automatic refunds are enabled for self-service cancellations,
+                            the system will try to refund the money automatically.
+                        {% endblocktrans %}
+                        {% blocktrans trimmed %}
+                            Refunds can be issued as a gift card if the respective option is set, but there is no customer choice between
+                            gift card and direct refund.
+                        {% endblocktrans %}
+                    </p>
+                </div>
             </fieldset>
         </div>
         <div class="form-group submit-group">

src/pretix/control/templates/pretixcontrol/event/invoicing.html

@@ -48,6 +48,7 @@
                 <legend>{% trans "Invoice customization" %}</legend>
                 {% bootstrap_field form.invoice_renderer layout="control" %}
                 {% bootstrap_field form.invoice_attendee_name layout="control" %}
+                {% bootstrap_field form.invoice_event_location layout="control" %}
                 {% bootstrap_field form.invoice_include_expire_date layout="control" %}
                 {% bootstrap_field form.invoice_introductory_text layout="control" %}
                 {% bootstrap_field form.invoice_additional_text layout="control" %}

src/pretix/control/templates/pretixcontrol/event/settings.html

@@ -204,7 +204,7 @@
                 {% bootstrap_field sform.logo_show_title layout="control" %}
                 {% bootstrap_field sform.og_image layout="control" %}
                 {% url "control:organizer.edit" organizer=request.organizer.slug as org_url %}
-                {% propagated request.event org_url "primary_color" "primary_font" "theme_color_success" "theme_color_danger" "theme_round_borders" %}
+                {% propagated request.event org_url "primary_color" "primary_font" "theme_color_success" "theme_color_danger" "theme_color_background" "theme_round_borders" %}
                     {% bootstrap_field sform.primary_color layout="control" %}
                     {% bootstrap_field sform.theme_color_success layout="control" %}
                     {% bootstrap_field sform.theme_color_danger layout="control" %}

src/pretix/control/templates/pretixcontrol/events/create_basics.html

@@ -17,7 +17,7 @@
                 <div class="help-block">
                     {% blocktrans trimmed %}
                         This is the address users can buy your tickets at. Should be short, only contain lowercase
-                        letters and numbers, and must be unique among your events. We recommend some kind of
+                        letters, numbers, dots, and dashes, and must be unique among your events. We recommend some kind of
                         abbreviation or a date with less than 10 characters that can be easily remembered, but you
                         can also choose to use a random value.
                     {% endblocktrans %}

src/pretix/control/templates/pretixcontrol/item/include_variations.html

@@ -77,6 +77,7 @@
                         {% bootstrap_field form.require_membership layout="control" %}
                         <div data-display-dependency="#{{ form.require_membership.id_for_label }}">
                             {% bootstrap_field form.require_membership_types layout="control" %}
+                            {% bootstrap_field form.require_membership_hidden layout="control" %}
                         </div>
                     {% endif %}
                 </div>
@@ -147,6 +148,7 @@
                         {% bootstrap_field formset.empty_form.require_membership layout="control" %}
                         <div data-display-dependency="#{{ formset.empty_form.require_membership.id_for_label }}">
                             {% bootstrap_field formset.empty_form.require_membership_types layout="control" %}
+                            {% bootstrap_field formset.empty_form.require_membership_hidden layout="control" %}
                         </div>
                     {% endif %}
                 </div>

src/pretix/control/templates/pretixcontrol/item/index.html

@@ -105,6 +105,7 @@
                             {% bootstrap_field form.require_membership layout="control" %}
                             <div data-display-dependency="#{{ form.require_membership.id_for_label }}">
                                 {% bootstrap_field form.require_membership_types layout="control" %}
+                                {% bootstrap_field form.require_membership_hidden layout="control" %}
                             </div>
                         {% endif %}
                         {% bootstrap_field form.allow_cancel layout="control" %}

src/pretix/control/templates/pretixcontrol/items/categories.html

@@ -24,6 +24,8 @@
             <a href="{% url "control:event.items.categories.add" organizer=request.event.organizer.slug event=request.event.slug %}" class="btn btn-default"><i class="fa fa-plus"></i> {% trans "Create a new category" %}
             </a>
         </p>
+        <form method="post">
+            {% csrf_token %}
         <div class="table-responsive">
             <table class="table table-hover">
                 <thead>
@@ -33,15 +35,16 @@
                     <th class="action-col-2"></th>
                 </tr>
                 </thead>
-                <tbody>
+                <tbody data-dnd-url="{% url "control:event.items.categories.reorder" organizer=request.event.organizer.slug event=request.event.slug %}">
                 {% for c in categories %}
-                    <tr>
+                    <tr data-dnd-id="{{ c.id }}">
                         <td>
                             <strong><a href="{% url "control:event.items.categories.edit" organizer=request.event.organizer.slug event=request.event.slug category=c.id %}">{{ c.internal_name|default:c.name }}</a></strong>
                         </td>
                         <td>
-                            <a href="{% url "control:event.items.categories.up" organizer=request.event.organizer.slug event=request.event.slug category=c.id %}" class="btn btn-default btn-sm {% if forloop.counter0 == 0 and is_paginated and not page_obj.has_previous %}disabled{% endif %}"><i class="fa fa-arrow-up"></i></a>
-                            <a href="{% url "control:event.items.categories.down" organizer=request.event.organizer.slug event=request.event.slug category=c.id %}" class="btn btn-default btn-sm {% if forloop.revcounter0 == 0 and is_paginated and not page_obj.has_next %}disabled{% endif %}"><i class="fa fa-arrow-down"></i></a>
+                            <button formaction="{% url "control:event.items.categories.up" organizer=request.event.organizer.slug event=request.event.slug category=c.id %}" class="btn btn-default btn-sm sortable-up"{% if forloop.counter0 == 0 and not page_obj.has_previous %} disabled{% endif %}><i class="fa fa-arrow-up"></i></button>
+                            <button formaction="{% url "control:event.items.categories.down" organizer=request.event.organizer.slug event=request.event.slug category=c.id %}" class="btn btn-default btn-sm sortable-down"{% if forloop.revcounter0 == 0 and not page_obj.has_next %} disabled{% endif %}><i class="fa fa-arrow-down"></i></button>
+                            <span class="dnd-container"></span>
                         </td>
                         <td class="text-right flip">
                             <a href="{% url "control:event.items.categories.edit" organizer=request.event.organizer.slug event=request.event.slug category=c.id %}" class="btn btn-default btn-sm"><i class="fa fa-edit"></i></a>
@@ -56,6 +59,7 @@
                 </tbody>
             </table>
         </div>
+        </form>
         {% include "pretixcontrol/pagination.html" %}
     {% endif %}
 {% endblock %}

src/pretix/control/templates/pretixcontrol/items/index.html

@@ -26,6 +26,8 @@
             <a href="{% url "control:event.items.add" organizer=request.event.organizer.slug event=request.event.slug %}"
                     class="btn btn-default"><i class="fa fa-plus"></i> {% trans "Create a new product" %}</a>
         </p>
+        <form method="post">
+            {% csrf_token %}
         <div class="table-responsive">
             <table class="table table-condensed table-hover">
                 <thead>
@@ -36,15 +38,16 @@
                     <th class="iconcol"></th>
                     <th class="iconcol"></th>
                     <th>{% trans "Category" %}</th>
-                    <th class="action-col-2"></th>
-                    <th class="action-col-2"></th>
+                    <th class="action-col-2"><span class="sr-only">Move</span></th>
+                    <th class="action-col-2"><span class="sr-only">Edit</span></th>
                 </tr>
                 </thead>
-                <tbody>
                 {% regroup items by category as cat_list %}
                 {% for c in cat_list %}
+                    <tbody data-dnd-url="{% url "control:event.items.reorder" organizer=request.event.organizer.slug event=request.event.slug %}">
                     {% for i in c.list %}
-                        <tr {% if not i.active %}class="row-muted"{% endif %}>
+                        {% if forloop.counter0 == 0 and i.category %}<tr class="sortable-disabled"><th colspan="8" scope="colgroup" class="text-muted">{{ i.category.name }}</th></tr>{% endif %}
+                        <tr data-dnd-id="{{ i.id }}" {% if not i.active %}class="row-muted"{% endif %}>
                             <td><strong>
                                 {% if not i.active %}<strike>{% endif %}
                                 <a href="{% url "control:event.item" organizer=request.event.organizer.slug event=request.event.slug item=i.id %}">{{ i }}</a>
@@ -105,8 +108,9 @@
                             </td>
                             <td>{% if i.category %}{{ i.category.name }}{% endif %}</td>
                             <td>
-                                <a href="{% url "control:event.items.up" organizer=request.event.organizer.slug event=request.event.slug item=i.id %}" class="btn btn-default btn-sm {% if forloop.counter0 == 0 and is_paginated and not page_obj.has_previous %}disabled{% endif %}"><i class="fa fa-arrow-up"></i></a>
-                                <a href="{% url "control:event.items.down" organizer=request.event.organizer.slug event=request.event.slug item=i.id %}" class="btn btn-default btn-sm {% if forloop.revcounter0 == 0 and is_paginated and not page_obj.has_next %}disabled{% endif %}"><i class="fa fa-arrow-down"></i></a>
+                                <button formaction="{% url "control:event.items.up" organizer=request.event.organizer.slug event=request.event.slug item=i.id %}" class="btn btn-default btn-sm sortable-up"{% if forloop.counter0 == 0 %} disabled{% endif %}><i class="fa fa-arrow-up"></i></button>
+                                <button formaction="{% url "control:event.items.down" organizer=request.event.organizer.slug event=request.event.slug item=i.id %}" class="btn btn-default btn-sm sortable-down"{% if forloop.revcounter0 == 0 %} disabled{% endif %}><i class="fa fa-arrow-down"></i></button>
+                                <span class="dnd-container"></span>
                             </td>
                             <td class="text-right flip col-actions">
                                 <a href="{% url "control:event.item" organizer=request.event.organizer.slug event=request.event.slug item=i.id %}" class="btn btn-default btn-sm"><i class="fa fa-edit"></i></a>
@@ -115,10 +119,11 @@
                             </td>
                         </tr>
                     {% endfor %}
+                    </tbody>
                 {% endfor %}
-                </tbody>
             </table>
         </div>
+        </form>
         {% include "pretixcontrol/pagination.html" %}
     {% endif %}
 {% endblock %}

src/pretix/control/templates/pretixcontrol/order/index.html

@@ -52,18 +52,22 @@
                         {% if order.status == 'n' or order.status == 'e' %}
                             <a href="{% url "control:event.order.transition" event=request.event.slug organizer=request.event.organizer.slug code=order.code %}?status=p"
                                     class="btn {% if overpaid >= 0 %}btn-success{% else %}btn-default{% endif %}">
+                                <span class="fa fa-check"></span>
                                 {% trans "Mark as paid" %}
                             </a>
                             <a href="{% url "control:event.order.extend" event=request.event.slug organizer=request.event.organizer.slug code=order.code %}" class="btn btn-default">
+                                <span class="fa fa-clock-o"></span>
                                 {% trans "Extend payment term" %}
                             </a>
                         {% endif %}
                         {% if order.cancel_allowed %}
                             <a href="{% url "control:event.order.transition" event=request.event.slug organizer=request.event.organizer.slug code=order.code %}?status=c" class="btn btn-default">
+                                <span class="fa fa-ban"></span>
                                 {% trans "Cancel order" %}
                             </a>
                         {% elif order.status == 'c' %}
                             <a href="{% url "control:event.order.reactivate" event=request.event.slug organizer=request.event.organizer.slug code=order.code %}" class="btn btn-default">
+                                <span class="fa fa-reply"></span>
                                 {% trans "Reactivate order" %}
                             </a>
                         {% endif %}
@@ -71,11 +75,17 @@
 
                     <a href="{% eventurl request.event "presale:event.order" order=order.code secret=order.secret %}"
                             class="btn btn-default" target="_blank">
+                        <span class="fa fa-eye"></span>
                         {% trans "View order as user" %}
                     </a>
                     <a href="{% url "control:event.order.mail_history" event=request.event.slug organizer=request.event.organizer.slug code=order.code %}" class="btn btn-default">
+                        <span class="fa fa-envelope-o"></span>
                         {% trans "View email history" %}
                     </a>
+                    <a href="{% url "control:event.order.transactions" event=request.event.slug organizer=request.event.organizer.slug code=order.code %}" class="btn btn-default">
+                        <span class="fa fa-list"></span>
+                        {% trans "View transaction history" %}
+                    </a>
                 </div>
             </div>
         </form>
@@ -234,7 +244,23 @@
                                 {% for i in invoices %}
                                     <a href="{% url "control:event.invoice.download" invoice=i.pk event=request.event.slug organizer=request.event.organizer.slug %}">
                                         {% if i.is_cancellation %}{% trans "Cancellation" context "invoice" %}{% else %}{% trans "Invoice" %}{% endif %}
-                                        {{ i.number }}</a> ({{ i.date|date:"SHORT_DATE_FORMAT" }})
+                                        {{ i.number }}</a>
+                                    ({{ i.date|date:"SHORT_DATE_FORMAT" }})
+                                    {% if i.sent_to_customer.year == 1970 %}
+                                        <span class="fa-stack fa-stack-small" data-toggle="tooltip" title="{% trans "We don't know if this invoice was emailed to the customer since it was created before our system tracked this information" %}">
+                                            <span class="fa fa-background fa-envelope text-muted fa-stack-1x"></span>
+                                            <span class="fa fa-question fa-stack-1x fa-stack-shifted"></span>
+                                        </span>
+                                    {% elif i.sent_to_customer %}
+                                        <span class="fa-stack fa-stack-small" data-toggle="tooltip" title="{% trans "Invoice was emailed to customer" %}">
+                                            <span class="fa fa-background fa-envelope text-muted fa-stack-1x"></span>
+                                            <span class="fa fa-check text-success fa-stack-1x fa-stack-shifted"></span>
+                                        </span>
+                                    {% else %}
+                                        <span class="fa-stack fa-stack-small" data-toggle="tooltip" title="{% trans "Invoice was not yet emailed to customer" %}">
+                                            <span class="fa fa-background fa-envelope text-muted fa-stack-1x"></span>
+                                        </span>
+                                    {% endif %}
                                     {% if not i.canceled %}
                                         {% if request.event.settings.invoice_regenerate_allowed %}
                                             <form class="form-inline helper-display-inline" method="post"
@@ -268,6 +294,12 @@
                                         <br/>
                                     {% endif %}
                                 {% endfor %}
+                                {% if invoices_send_link %}
+                                    <br/>
+                                    <a class="btn btn-default btn-xs" href="{{ invoices_send_link }}">
+                                        {% trans "Email invoices" %}
+                                    </a>
+                                {% endif %}
                                 {% if can_generate_invoice %}
                                     <br/>
                                     <form class="form-inline helper-display-inline" method="post"

src/pretix/control/templates/pretixcontrol/order/mail_history.html

@@ -28,7 +28,7 @@
                             <br/><span class="fa fa-fw fa-envelope-o"></span> {{ log.parsed_data.recipient }}
                         {% endif %}
                     </p>
-                    {% if log.parsed_data.subject.items %}
+                    {% if log.parsed_data.message.items %}
                         <div class="alert alert-info">
                             {% blocktrans trimmed %}
                                 This email has been sent with an older version of pretix. We are therefore not able to

src/pretix/control/templates/pretixcontrol/order/sendmail.html

@@ -18,6 +18,10 @@
             {% bootstrap_field form.sendto layout='horizontal' %}
             {% bootstrap_field form.subject layout='horizontal' %}
             {% bootstrap_field form.message layout='horizontal' %}
+            {% bootstrap_field form.attach_tickets layout='horizontal' %}
+            {% if form.attach_invoices %}
+                {% bootstrap_field form.attach_invoices layout='horizontal' %}
+            {% endif %}
             {% if request.method == "POST" %}
                 <fieldset>
                     <legend>{% trans "E-mail preview" %}</legend>

src/pretix/control/templates/pretixcontrol/order/transactions.html

@@ -0,0 +1,81 @@
+{% extends "pretixcontrol/event/base.html" %}
+{% load i18n %}
+{% load bootstrap3 %}
+{% load money %}
+{% block title %}{% trans "Transaction history" %}{% endblock %}
+{% block content %}
+    <h1>
+        {% trans "Transaction history" %}
+        <a class="btn btn-link btn-lg"
+                href="{% url "control:event.order" event=request.event.slug organizer=request.event.organizer.slug code=order.code %}">
+            {% blocktrans trimmed with order=order.code %}
+                Back to order {{ order }}
+            {% endblocktrans %}
+        </a>
+    </h1>
+    <table class="table table-condensed table-hover">
+        <thead>
+        <tr>
+            <th>{% trans "Date" %}</th>
+            <th>{% trans "Product" %}</th>
+            <th class="text-right flip">{% trans "Tax rate" %}</th>
+            <th class="text-right flip">{% trans "Quantity" %}</th>
+            <th class="text-right flip">{% trans "Single price" %}</th>
+            <th class="text-right flip">{% trans "Total tax value" %}</th>
+            <th class="text-right flip">{% trans "Total price" %}</th>
+        </tr>
+        </thead>
+        <tbody>
+        {% for t in transactions %}
+            <tr class="{% if t.count < 0 %}text-danger{% endif %}">
+                <td>
+                    {{ t.datetime|date:"SHORT_DATETIME_FORMAT" }}
+                    {% if t.migrated %}
+                        <span class="fa fa-warning text-warning"
+                                data-toggle="tooltip"
+                                title="{% trans 'This order was created before we introduced this table, therefore this data might be inaccurate.' %}"
+                        ></span>
+                    {% endif %}
+                </td>
+                <td>
+                    {% if t.item %}
+                        {{ t.item }}
+                        {% if t.variation %}
+                            – {{ t.variation }}
+                        {% endif %}
+                    {% endif %}
+                    {% if t.fee_type %}
+                        {{ t.get_fee_type_display }}
+                    {% endif %}
+                    {% if t.subevent %}
+                        <br>{{ t.subevent }}
+                    {% endif %}
+                </td>
+                <td class="text-right flip">{{ t.tax_rate }} %</td>
+                <td class="text-right flip">{{ t.count }} &times;</td>
+                <td class="text-right flip">{{ t.price|money:request.event.currency }}</td>
+                <td class="text-right flip">{{ t.full_tax_value|money:request.event.currency }}</td>
+                <td class="text-right flip">{{ t.full_price|money:request.event.currency }}</td>
+            </tr>
+        {% endfor %}
+        </tbody>
+    <tfoot>
+    <tr class="{% if t.count < 0 %}text-danger{% endif %}">
+        <td>
+            <strong>{% trans "Sum" %}</strong>
+        </td>
+        <td>
+        </td>
+        <td></td>
+        <td class="text-right flip">
+            <strong>
+                {{ sums.count }}
+            </strong>
+        </td>
+        <td></td>
+        <td class="text-right flip"><strong>{{ sums.full_tax_value|money:request.event.currency }}</strong></td>
+        <td class="text-right flip"><strong>{{ sums.full_price|money:request.event.currency }}</strong></td>
+    </tr>
+    </tfoot>
+    </table>
+{% endblock %}

src/pretix/control/templates/pretixcontrol/organizers/edit.html

@@ -82,6 +82,49 @@
                         {% bootstrap_field sform.giftcard_expiry_years layout="control" %}
                         {% bootstrap_field sform.giftcard_length layout="control" %}
                     </fieldset>
+                    <fieldset>
+                        <legend>{% trans "Privacy" %}</legend>
+                        {% bootstrap_field sform.privacy_url layout="control" %}
+                        <div class="alert alert-legal">
+                            <p>
+                                {% blocktrans trimmed %}
+                                    Some jurisdictions, including the European Union, require user consent before you
+                                    are allowed to use cookies or similar technology for analytics, tracking, payment,
+                                    or similar purposes.
+                                {% endblocktrans %}
+                            </p>
+                            <p>
+                                {% blocktrans trimmed %}
+                                    pretix itself only ever sets cookies that are required to provide the service
+                                    requested by the user or to maintain an appropriate level of security. Therefore,
+                                    cookies set by pretix itself do not require consent in all jurisdictions that we
+                                    are aware of.
+                                {% endblocktrans %}
+                            </p>
+                            <p>
+                                {% blocktrans trimmed %}
+                                    Therefore, the settings on this page will <strong>only</strong> have an affect
+                                    if you use <strong>plugins</strong> that require additional cookies
+                                    <strong>and</strong> participate in our cookie consent mechanism.
+                                {% endblocktrans %}
+                            </p>
+                            <p>
+                                <strong>{% blocktrans trimmed %}
+                                    Ultimately, it is your responsibility to make sure you comply with all relevant
+                                    laws. We try to help by providing these settings, but we cannot assume liability
+                                    since we do not know the exact configuration of your pretix usage, the legal details
+                                    in your specific jurisdiction, or the agreements you have with third parties such as
+                                    payment or tracking providers.
+                                {% endblocktrans %}</strong>
+                            </p>
+                        </div>
+                        {% bootstrap_field sform.cookie_consent layout="control" %}
+                        {% bootstrap_field sform.cookie_consent_dialog_title layout="control" %}
+                        {% bootstrap_field sform.cookie_consent_dialog_text layout="control" %}
+                        {% bootstrap_field sform.cookie_consent_dialog_text_secondary layout="control" %}
+                        {% bootstrap_field sform.cookie_consent_dialog_button_yes layout="control" %}
+                        {% bootstrap_field sform.cookie_consent_dialog_button_no layout="control" %}
+                    </fieldset>
                     <fieldset>
                         <legend>{% trans "Invoices" %}</legend>
                         {% bootstrap_field sform.invoice_regenerate_allowed layout="control" %}

src/pretix/control/templates/pretixcontrol/users/create.html

@@ -19,6 +19,7 @@
             {% bootstrap_field form.email layout='control' %}
             {% bootstrap_field form.new_pw layout='control' %}
             {% bootstrap_field form.new_pw_repeat layout='control' %}
+            {% bootstrap_field form.needs_password_change layout='control' %}
         </fieldset>
         <div class="form-group submit-group">
             <button type="submit" class="btn btn-primary btn-save">

src/pretix/control/templates/pretixcontrol/users/form.html

@@ -45,6 +45,7 @@
                     {% endif %}
                     {% bootstrap_field form.last_login layout='control' %}
                     {% bootstrap_field form.require_2fa layout='control' %}
+                    {% bootstrap_field form.needs_password_change layout='control' %}
                 </fieldset>
                 <fieldset>
                     <legend>{% trans "Team memberships" %}</legend>

src/pretix/control/urls.py

@@ -239,6 +239,7 @@ urlpatterns = [
         re_path(r'^items/(?P<item>\d+)/$', item.ItemUpdateGeneral.as_view(), name='event.item'),
         re_path(r'^items/(?P<item>\d+)/up$', item.item_move_up, name='event.items.up'),
         re_path(r'^items/(?P<item>\d+)/down$', item.item_move_down, name='event.items.down'),
+        re_path(r'^items/reorder$', item.reorder_items, name='event.items.reorder'),
         re_path(r'^items/(?P<item>\d+)/delete$', item.ItemDelete.as_view(), name='event.items.delete'),
         re_path(r'^items/typeahead/meta/$', typeahead.item_meta_values, name='event.items.meta.typeahead'),
         re_path(r'^items/select2$', typeahead.items_select2, name='event.items.select2'),
@@ -250,6 +251,7 @@ urlpatterns = [
         re_path(r'^categories/(?P<category>\d+)/up$', item.category_move_up, name='event.items.categories.up'),
         re_path(r'^categories/(?P<category>\d+)/down$', item.category_move_down,
                 name='event.items.categories.down'),
+        re_path(r'^categories/reorder$', item.reorder_categories, name='event.items.categories.reorder'),
         re_path(r'^categories/(?P<category>\d+)/$', item.CategoryUpdate.as_view(),
                 name='event.items.categories.edit'),
         re_path(r'^categories/add$', item.CategoryCreate.as_view(), name='event.items.categories.add'),
@@ -343,6 +345,7 @@ urlpatterns = [
         re_path(r'^orders/(?P<code>[0-9A-Z]+)/cancellationrequests/(?P<req>\d+)/delete$',
                 orders.OrderCancellationRequestDelete.as_view(),
                 name='event.order.cancellationrequests.delete'),
+        re_path(r'^orders/(?P<code>[0-9A-Z]+)/transactions/$', orders.OrderTransactions.as_view(), name='event.order.transactions'),
         re_path(r'^orders/(?P<code>[0-9A-Z]+)/$', orders.OrderDetail.as_view(), name='event.order'),
         re_path(r'^invoice/(?P<invoice>[^/]+)$', orders.InvoiceDownload.as_view(),
                 name='event.invoice.download'),

src/pretix/control/views/item.py

@@ -53,6 +53,7 @@ from django.urls import resolve, reverse
 from django.utils.functional import cached_property
 from django.utils.timezone import now
 from django.utils.translation import gettext, gettext_lazy as _
+from django.views.decorators.http import require_http_methods
 from django.views.generic import ListView
 from django.views.generic.detail import DetailView, SingleObjectMixin
 from django.views.generic.edit import DeleteView
@@ -101,7 +102,8 @@ class ItemList(ListView):
         ).annotate(
             var_count=Count('variations')
         ).prefetch_related("category").order_by(
-            'category__position', 'category', 'position'
+            F('category__position').desc(nulls_first=True),
+            'category', 'position'
         )
 
     def get_context_data(self, **kwargs):
@@ -138,6 +140,7 @@ def item_move(request, item, up=True):
 
 
 @event_permission_required("can_change_items")
+@require_http_methods(["POST"])
 def item_move_up(request, organizer, event, item):
     item_move(request, item, up=True)
     return redirect('control:event.items',
@@ -146,6 +149,7 @@ def item_move_up(request, organizer, event, item):
 
 
 @event_permission_required("can_change_items")
+@require_http_methods(["POST"])
 def item_move_down(request, organizer, event, item):
     item_move(request, item, up=False)
     return redirect('control:event.items',
@@ -153,6 +157,38 @@ def item_move_down(request, organizer, event, item):
                     event=request.event.slug)
 
 
+@transaction.atomic
+@event_permission_required("can_change_items")
+@require_http_methods(["POST"])
+def reorder_items(request, organizer, event):
+    try:
+        ids = json.loads(request.body.decode('utf-8'))['ids']
+    except (JSONDecodeError, KeyError, ValueError):
+        return HttpResponseBadRequest("expected JSON: {ids:[]}")
+
+    input_items = list(request.event.items.filter(id__in=[i for i in ids if i.isdigit()]))
+
+    if len(input_items) != len(ids):
+        raise Http404(_("Some of the provided item ids are invalid."))
+
+    item_categories = {i.category_id for i in input_items}
+    if len(item_categories) > 1:
+        raise Http404(_("You cannot reorder items spanning different categories."))
+
+    # get first and only category
+    item_category = next(iter(item_categories))
+    if len(input_items) != request.event.items.filter(category=item_category).count():
+        raise Http404(_("Not all items have been selected."))
+
+    for i in input_items:
+        pos = ids.index(str(i.pk))
+        if pos != i.position:  # Save unneccessary UPDATE queries
+            i.position = pos
+            i.save(update_fields=['position'])
+
+    return HttpResponse()
+
+
 class CategoryDelete(EventPermissionRequiredMixin, DeleteView):
     model = ItemCategory
     form_class = CategoryForm
@@ -307,6 +343,7 @@ def category_move(request, category, up=True):
 
 
 @event_permission_required("can_change_items")
+@require_http_methods(["POST"])
 def category_move_up(request, organizer, event, category):
     category_move(request, category, up=True)
     return redirect('control:event.items.categories',
@@ -315,6 +352,7 @@ def category_move_up(request, organizer, event, category):
 
 
 @event_permission_required("can_change_items")
+@require_http_methods(["POST"])
 def category_move_down(request, organizer, event, category):
     category_move(request, category, up=False)
     return redirect('control:event.items.categories',
@@ -322,6 +360,32 @@ def category_move_down(request, organizer, event, category):
                     event=request.event.slug)
 
 
+@transaction.atomic
+@event_permission_required("can_change_items")
+@require_http_methods(["POST"])
+def reorder_categories(request, organizer, event):
+    try:
+        ids = json.loads(request.body.decode('utf-8'))['ids']
+    except (JSONDecodeError, KeyError, ValueError):
+        return HttpResponseBadRequest("expected JSON: {ids:[]}")
+
+    input_categories = list(request.event.categories.filter(id__in=[i for i in ids if i.isdigit()]))
+
+    if len(input_categories) != len(ids):
+        raise Http404(_("Some of the provided category ids are invalid."))
+
+    if len(input_categories) != request.event.categories.count():
+        raise Http404(_("Not all categories have been selected."))
+
+    for c in input_categories:
+        pos = ids.index(str(c.pk))
+        if pos != c.position:  # Save unneccessary UPDATE queries
+            c.position = pos
+            c.save(update_fields=['position'])
+
+    return HttpResponse()
+
+
 FakeQuestion = namedtuple(
     'FakeQuestion', 'id question position required'
 )
@@ -337,10 +401,10 @@ class QuestionList(ListView):
 
     def get_context_data(self, **kwargs):
         ctx = super().get_context_data(**kwargs)
-        ctx['questions'] = list(ctx['questions'])
+        questions = []
 
         if self.request.event.settings.attendee_names_asked:
-            ctx['questions'].append(
+            questions.append(
                 FakeQuestion(
                     id='attendee_name_parts',
                     question=_('Attendee name'),
@@ -352,7 +416,7 @@ class QuestionList(ListView):
             )
 
         if self.request.event.settings.attendee_emails_asked:
-            ctx['questions'].append(
+            questions.append(
                 FakeQuestion(
                     id='attendee_email',
                     question=_('Attendee email'),
@@ -363,8 +427,8 @@ class QuestionList(ListView):
                 )
             )
 
-        if self.request.event.settings.attendee_emails_asked:
-            ctx['questions'].append(
+        if self.request.event.settings.attendee_company_asked:
+            questions.append(
                 FakeQuestion(
                     id='company',
                     question=_('Company'),
@@ -376,7 +440,7 @@ class QuestionList(ListView):
             )
 
         if self.request.event.settings.attendee_addresses_asked:
-            ctx['questions'].append(
+            questions.append(
                 FakeQuestion(
                     id='street',
                     question=_('Street'),
@@ -386,7 +450,7 @@ class QuestionList(ListView):
                     required=self.request.event.settings.attendee_addresses_required,
                 )
             )
-            ctx['questions'].append(
+            questions.append(
                 FakeQuestion(
                     id='zipcode',
                     question=_('ZIP code'),
@@ -396,7 +460,7 @@ class QuestionList(ListView):
                     required=self.request.event.settings.attendee_addresses_required,
                 )
             )
-            ctx['questions'].append(
+            questions.append(
                 FakeQuestion(
                     id='city',
                     question=_('City'),
@@ -406,7 +470,7 @@ class QuestionList(ListView):
                     required=self.request.event.settings.attendee_addresses_required,
                 )
             )
-            ctx['questions'].append(
+            questions.append(
                 FakeQuestion(
                     id='country',
                     question=_('Country'),
@@ -417,24 +481,29 @@ class QuestionList(ListView):
                 )
             )
 
-        ctx['questions'].sort(key=lambda q: q.position)
+        questions += list(ctx['questions'])
+        questions.sort(key=lambda q: q.position)
+        ctx['questions'] = questions
         return ctx
 
 
 @transaction.atomic
 @event_permission_required("can_change_items")
+@require_http_methods(["POST"])
 def reorder_questions(request, organizer, event):
     try:
         ids = json.loads(request.body.decode('utf-8'))['ids']
     except (JSONDecodeError, KeyError, ValueError):
         return HttpResponseBadRequest("expected JSON: {ids:[]}")
 
-    input_questions = request.event.questions.filter(id__in=[i for i in ids if i.isdigit()])
+    # filter system_questions - normal questions are int/digit, system_questions strings
+    custom_question_ids = [i for i in ids if i.isdigit()]
+    input_questions = list(request.event.questions.filter(id__in=custom_question_ids))
 
-    if input_questions.count() != len([i for i in ids if i.isdigit()]):
+    if len(input_questions) != len(custom_question_ids):
         raise Http404(_("Some of the provided question ids are invalid."))
 
-    if input_questions.count() != request.event.questions.count():
+    if len(input_questions) != request.event.questions.count():
         raise Http404(_("Not all questions have been selected."))
 
     for q in input_questions:

src/pretix/control/views/orders.py

@@ -42,7 +42,6 @@ from datetime import datetime, time, timedelta
 from decimal import Decimal, DecimalException
 from urllib.parse import quote, urlencode
 
-import vat_moss.id
 from django import forms
 from django.conf import settings
 from django.contrib import messages
@@ -50,7 +49,7 @@ from django.core.exceptions import ValidationError
 from django.core.files import File
 from django.db import transaction
 from django.db.models import (
-    Count, Exists, IntegerField, OuterRef, Prefetch, ProtectedError, Q,
+    Count, Exists, F, IntegerField, OuterRef, Prefetch, ProtectedError, Q,
     Subquery, Sum,
 )
 from django.forms import formset_factory
@@ -65,7 +64,7 @@ from django.utils.formats import date_format, get_format
 from django.utils.functional import cached_property
 from django.utils.http import is_safe_url
 from django.utils.timezone import make_aware, now
-from django.utils.translation import gettext, gettext_lazy as _
+from django.utils.translation import gettext, gettext_lazy as _, ngettext
 from django.views.generic import (
     DetailView, FormView, ListView, TemplateView, View,
 )
@@ -83,7 +82,7 @@ from pretix.base.models import (
 from pretix.base.models.orders import (
     CancellationRequest, OrderFee, OrderPayment, OrderPosition, OrderRefund,
 )
-from pretix.base.models.tax import cc_to_vat_prefix, is_eu_country
+from pretix.base.models.tax import ask_for_vat_id
 from pretix.base.payment import PaymentException
 from pretix.base.secrets import assign_ticket_secret
 from pretix.base.services import tickets
@@ -103,6 +102,9 @@ from pretix.base.services.orders import (
     notify_user_changed_order, reactivate_order,
 )
 from pretix.base.services.stats import order_overview
+from pretix.base.services.tax import (
+    VATIDFinalError, VATIDTemporaryError, validate_vat_id,
+)
 from pretix.base.services.tickets import generate
 from pretix.base.signals import (
     order_modified, register_data_exporters, register_ticket_outputs,
@@ -313,6 +315,27 @@ class OrderDetail(OrderView):
         ctx['download_buttons'] = self.download_buttons
         ctx['payment_refund_sum'] = self.order.payment_refund_sum
         ctx['pending_sum'] = self.order.pending_sum
+
+        unsent_invoices = [ii.pk for ii in ctx['invoices'] if not ii.sent_to_customer]
+        if unsent_invoices:
+            ctx['invoices_send_link'] = reverse('control:event.order.sendmail', kwargs={
+                'event': self.request.event.slug,
+                'organizer': self.request.event.organizer.slug,
+                'code': self.order.code
+            }) + '?' + urlencode({
+                'subject': ngettext('Your invoice', 'Your invoices', len(unsent_invoices)),
+                'message': ngettext(
+                    'Hello,\n\nplease find your invoice attached to this email.\n\n'
+                    'Your {event} team',
+                    'Hello,\n\nplease find your invoices attached to this email.\n\n'
+                    'Your {event} team',
+                    len(unsent_invoices)
+                ).format(
+                    event="{event}",
+                ),
+                'attach_invoices': unsent_invoices
+            }, doseq=True)
+
         return ctx
 
     @cached_property
@@ -380,6 +403,23 @@ class OrderDetail(OrderView):
         }
 
 
+class OrderTransactions(OrderView):
+    template_name = 'pretixcontrol/order/transactions.html'
+    permission = 'can_view_orders'
+
+    def get_context_data(self, **kwargs):
+        ctx = super().get_context_data(**kwargs)
+        ctx['transactions'] = self.order.transactions.select_related(
+            'item', 'variation', 'subevent'
+        ).order_by('datetime')
+        ctx['sums'] = self.order.transactions.aggregate(
+            count=Sum('count'),
+            full_price=Sum(F('count') * F('price')),
+            full_tax_value=Sum(F('count') * F('tax_value')),
+        )
+        return ctx
+
+
 class OrderDownload(AsyncAction, OrderView):
     task = generate
     permission = 'can_view_orders'
@@ -1137,6 +1177,19 @@ class OrderTransition(OrderView):
         to = self.request.POST.get('status', '')
         if self.order.status in (Order.STATUS_PENDING, Order.STATUS_EXPIRED) and to == 'p' and self.mark_paid_form.is_valid():
             ps = self.mark_paid_form.cleaned_data['amount']
+
+            if ps == Decimal('0.00') and self.order.pending_sum <= Decimal('0.00'):
+                p = self.order.payments.filter(state=OrderPayment.PAYMENT_STATE_CONFIRMED).last()
+                if p:
+                    p._mark_order_paid(
+                        user=self.request.user,
+                        send_mail=self.mark_paid_form.cleaned_data['send_email'],
+                        force=self.mark_paid_form.cleaned_data.get('force', False),
+                        payment_refund_sum=self.order.payment_refund_sum,
+                    )
+                    messages.success(self.request, _('The order has been marked as paid.'))
+                    return redirect(self.get_order_url())
+
             try:
                 p = self.order.payments.get(
                     state__in=(OrderPayment.PAYMENT_STATE_PENDING, OrderPayment.PAYMENT_STATE_CREATED),
@@ -1292,26 +1345,18 @@ class OrderCheckVATID(OrderView):
                 messages.error(self.request, _('No country specified.'))
                 return redirect(self.get_order_url())
 
-            if not is_eu_country(ia.country):
-                messages.error(self.request, _('VAT ID could not be checked since a non-EU country has been '
-                                               'specified.'))
-                return redirect(self.get_order_url())
-
-            if ia.vat_id[:2] != cc_to_vat_prefix(str(ia.country)):
-                messages.error(self.request, _('Your VAT ID does not match the selected country.'))
+            if not ask_for_vat_id(ia.country):
+                messages.error(self.request, _('VAT ID could not be checked since this country is not supported.'))
                 return redirect(self.get_order_url())
 
             try:
-                result = vat_moss.id.validate(ia.vat_id)
-                if result:
-                    country_code, normalized_id, company_name = result
-                    ia.vat_id_validated = True
-                    ia.vat_id = normalized_id
-                    ia.save()
-            except vat_moss.errors.InvalidError:
-                messages.error(self.request, _('This VAT ID is not valid.'))
-            except vat_moss.errors.WebServiceUnavailableError:
-                logger.exception('VAT ID checking failed for country {}'.format(ia.country))
+                normalized_id = validate_vat_id(ia.vat_id, str(ia.country))
+                ia.vat_id_validated = True
+                ia.vat_id = normalized_id
+                ia.save()
+            except VATIDFinalError as e:
+                messages.error(self.request, e.message)
+            except VATIDTemporaryError:
                 messages.error(self.request, _('The VAT ID could not be checked, as the VAT checking service of '
                                                'the country is currently not available.'))
             else:
@@ -1954,6 +1999,8 @@ class OrderSendMail(EventPermissionRequiredMixin, OrderViewMixin, FormView):
             kwargs['initial']['subject'] = self.request.GET.get('subject')
         if self.request.GET.get('message'):
             kwargs['initial']['message'] = self.request.GET.get('message')
+        if self.request.GET.getlist('attach_invoices'):
+            kwargs['initial']['attach_invoices'] = self.order.invoices.filter(pk__in=self.request.GET.getlist('attach_invoices'))
         return kwargs
 
     def form_invalid(self, form):
@@ -1982,7 +2029,9 @@ class OrderSendMail(EventPermissionRequiredMixin, OrderViewMixin, FormView):
                 order.send_mail(
                     form.cleaned_data['subject'], email_template,
                     email_context, 'pretix.event.order.email.custom_sent',
-                    self.request.user, auto_email=False
+                    self.request.user, auto_email=False,
+                    attach_tickets=form.cleaned_data.get('attach_tickets', False),
+                    invoices=form.cleaned_data.get('attach_invoices', []),
                 )
                 messages.success(self.request,
                                  _('Your message has been queued and will be sent to {}.'.format(order.email)))
@@ -2047,7 +2096,8 @@ class OrderPositionSendMail(OrderSendMail):
                     email_template,
                     email_context,
                     'pretix.event.order.position.email.custom_sent',
-                    self.request.user
+                    self.request.user,
+                    attach_tickets=form.cleaned_data.get('attach_tickets', False),
                 )
                 messages.success(self.request,
                                  _('Your message has been queued and will be sent to {}.'.format(position.attendee_email)))

src/pretix/control/views/subevents.py

@@ -273,6 +273,7 @@ class SubEventEditorMixin(MetaDataEditorMixin):
                     'size': q.size,
                     'name': q.name,
                     'release_after_exit': q.release_after_exit,
+                    'ignore_for_event_availability': q.ignore_for_event_availability,
                     'itemvars': [str(i.pk) for i in q.items.all()] + [
                         '{}-{}'.format(v.item_id, v.pk) for v in q.variations.all()
                     ]

src/pretix/control/views/user.py

@@ -226,6 +226,7 @@ class UserSettings(UpdateView):
         msgs = []
 
         if 'new_pw' in form.changed_data:
+            self.request.user.needs_password_change = False
             msgs.append(_('Your password has been changed.'))
 
         if 'email' in form.changed_data:
@@ -243,6 +244,8 @@ class UserSettings(UpdateView):
         return sup
 
     def get_success_url(self):
+        if "next" in self.request.GET and url_has_allowed_host_and_scheme(self.request.GET.get("next"), allowed_hosts=None):
+            return self.request.GET.get("next")
         return reverse('control:user.settings')
 
 

src/pretix/helpers/daterange.py

@@ -33,36 +33,67 @@
 # License for the specific language governing permissions and limitations under the License.
 
 from django.template.defaultfilters import date as _date
+from django.utils.html import format_html
 from django.utils.translation import get_language, gettext_lazy as _
 
 
-def daterange(df, dt):
+def daterange(df, dt, as_html=False):
     lng = get_language()
+    if df.year == dt.year and df.month == dt.month and df.day == dt.day:
+        if as_html:
+            base_format = format_html("<time datetime=\"{}\">{{}}</time>", _date(df, "Y-m-d"))
+        else:
+            base_format = "{}"
+    else:
+        if as_html:
+            base_format = format_html("<time datetime=\"{}\">{{}}</time>{{}}<time datetime=\"{}\">{{}}</time>", _date(df, "Y-m-d"), _date(dt, "Y-m-d"))
+        else:
+            base_format = "{}{}{}"
 
     if lng.startswith("de"):
         if df.year == dt.year and df.month == dt.month and df.day == dt.day:
-            return "{}".format(_date(df, "j. F Y"))
+            return format_html(base_format, _date(df, "j. F Y"))
         elif df.year == dt.year and df.month == dt.month:
-            return "{}.–{}".format(_date(df, "j"), _date(dt, "j. F Y"))
+            return format_html(base_format, _date(df, "j."), "–", _date(dt, "j. F Y"))
         elif df.year == dt.year:
-            return "{} – {}".format(_date(df, "j. F"), _date(dt, "j. F Y"))
+            return format_html(base_format, _date(df, "j. F"), " – ", _date(dt, "j. F Y"))
     elif lng.startswith("en"):
         if df.year == dt.year and df.month == dt.month and df.day == dt.day:
-            return "{}".format(_date(df, "N jS, Y"))
+            return format_html(base_format, _date(df, "N jS, Y"))
         elif df.year == dt.year and df.month == dt.month:
-            return "{} – {}".format(_date(df, "N jS"), _date(dt, "jS, Y"))
+            return format_html(base_format, _date(df, "N jS"), " – ", _date(dt, "jS, Y"))
         elif df.year == dt.year:
-            return "{} – {}".format(_date(df, "N jS"), _date(dt, "N jS, Y"))
+            return format_html(base_format, _date(df, "N jS"), " – ", _date(dt, "N jS, Y"))
     elif lng.startswith("es"):
         if df.year == dt.year and df.month == dt.month and df.day == dt.day:
-            return "{}".format(_date(df, "DATE_FORMAT"))
+            return format_html(base_format, _date(df, "DATE_FORMAT"))
         elif df.year == dt.year and df.month == dt.month:
-            return "{} - {} de {} de {}".format(_date(df, "j"), _date(dt, "j"), _date(dt, "F"), _date(dt, "Y"))
+            return format_html(
+                base_format,
+                _date(df, "j"),
+                " - ",
+                "{} de {} de {}".format(_date(dt, "j"), _date(dt, "F"), _date(dt, "Y"))
+            )
         elif df.year == dt.year:
-            return "{} de {} - {} de {} de {}".format(_date(df, "j"), _date(df, "F"), _date(dt, "j"), _date(dt, "F"), _date(dt, "Y"))
+            return format_html(
+                base_format,
+                "{} de {}".format(_date(df, "j"), _date(df, "F")),
+                " - ",
+                "{} de {} de {}".format(_date(dt, "j"), _date(dt, "F"), _date(dt, "Y"))
+            )
 
     if df.year == dt.year and df.month == dt.month and df.day == dt.day:
-        return _date(df, "DATE_FORMAT")
+        return format_html(base_format, _date(df, "DATE_FORMAT"))
+
+    if as_html:
+        base_format = "<time datetime=\"{}\">{}</time>"
+        return format_html(
+            "{date_from} – {date_to}",
+            date_from=format_html(base_format, _date(df, "Y-m-d"), _date(df, "DATE_FORMAT")),
+            date_to=format_html(base_format, _date(dt, "Y-m-d"), _date(dt, "DATE_FORMAT")),
+        )
+
     return _("{date_from} – {date_to}").format(
-        date_from=_date(df, "DATE_FORMAT"), date_to=_date(dt, "DATE_FORMAT")
+        date_from=_date(df, "DATE_FORMAT"),
+        date_to=_date(dt, "DATE_FORMAT"),
     )

src/pretix/locale/ar/LC_MESSAGES/djangojs.po

@@ -7,7 +7,7 @@ msgid ""
 msgstr ""
 "Project-Id-Version: PACKAGE VERSION\n"
 "Report-Msgid-Bugs-To: \n"
-"POT-Creation-Date: 2021-09-27 18:49+0000\n"
+"POT-Creation-Date: 2021-10-29 10:10+0000\n"
 "PO-Revision-Date: 2021-09-15 11:22+0000\n"
 "Last-Translator: Mohamed Tawfiq <mtawfiq@wafyapp.com>\n"
 "Language-Team: Arabic <https://translate.pretix.eu/projects/pretix/pretix-js/"
@@ -199,6 +199,13 @@ msgstr "تذاكر سارية المفعول"
 msgid "Currently inside"
 msgstr "حاليا بالداخل"
 
+#: pretix/static/lightbox/js/lightbox.js:96
+#, fuzzy
+#| msgctxt "widget"
+#| msgid "Close"
+msgid "close"
+msgstr "إغلاق"
+
 #: pretix/static/pretixbase/js/asynctask.js:43
 #: pretix/static/pretixbase/js/asynctask.js:119
 msgid ""
@@ -261,7 +268,7 @@ msgstr ""
 "نعمل الآن على ارسال طلبك إلى الخادم، إذا أستغرقت العملية أكثر من دقيقة، يرجى "
 "التحقق من اتصالك بالإنترنت ثم أعد تحميل الصفحة مرة أخرى."
 
-#: pretix/static/pretixbase/js/asynctask.js:262
+#: pretix/static/pretixbase/js/asynctask.js:264
 #: pretix/static/pretixcontrol/js/ui/main.js:34
 msgid "Close message"
 msgstr "أغلق الرسالة"
@@ -490,17 +497,25 @@ msgstr[3] "أيام"
 msgstr[4] "أيام عديدة"
 msgstr[5] "أخرى"
 
-#: pretix/static/pretixpresale/js/ui/cart.js:39
-msgid "The items in your cart are no longer reserved for you."
+#: pretix/static/pretixpresale/js/ui/cart.js:43
+#, fuzzy
+#| msgid "The items in your cart are no longer reserved for you."
+msgid ""
+"The items in your cart are no longer reserved for you. You can still "
+"complete your order as long as they’re available."
 msgstr "لم تعد العناصر الموجودة في عربة التسوق الخاصة بك محجوزة لك."
 
-#: pretix/static/pretixpresale/js/ui/cart.js:41
+#: pretix/static/pretixpresale/js/ui/cart.js:45
 msgid "Cart expired"
 msgstr "انتهت صلاحية عربة التسوق"
 
-#: pretix/static/pretixpresale/js/ui/cart.js:46
-msgid "The items in your cart are reserved for you for one minute."
-msgid_plural "The items in your cart are reserved for you for {num} minutes."
+#: pretix/static/pretixpresale/js/ui/cart.js:50
+#, fuzzy
+#| msgid "The items in your cart are reserved for you for one minute."
+#| msgid_plural ""
+#| "The items in your cart are reserved for you for {num} minutes."
+msgid "The items in your cart are reserved for you for one minute."
+msgid_plural "The items in your cart are reserved for you for {num} minutes."
 msgstr[0] "سيتم حجز العناصر لك في سلة التسوق لمدة {num}."
 msgstr[1] "سيتم حجز العناصر لك في سلة التسوق لمدة دقيقة {num}."
 msgstr[2] "سيتم حجز العناصر لك في سلة التسوق لمدة دقيقتين {num}."
@@ -520,20 +535,20 @@ msgstr "ستسترد %(currency)%(amount)"
 msgid "Please enter the amount the organizer can keep."
 msgstr "الرجاء إدخال المبلغ الذي يمكن للمنظم الاحتفاظ به."
 
-#: pretix/static/pretixpresale/js/ui/main.js:309
+#: pretix/static/pretixpresale/js/ui/main.js:335
 msgid "Please enter a quantity for one of the ticket types."
 msgstr "الرجاء إدخال عدد لأحد أنواع التذاكر."
 
-#: pretix/static/pretixpresale/js/ui/main.js:345
+#: pretix/static/pretixpresale/js/ui/main.js:371
 msgid "required"
 msgstr "مطلوب"
 
-#: pretix/static/pretixpresale/js/ui/main.js:447
-#: pretix/static/pretixpresale/js/ui/main.js:465
+#: pretix/static/pretixpresale/js/ui/main.js:473
+#: pretix/static/pretixpresale/js/ui/main.js:491
 msgid "Time zone:"
 msgstr "المنطقة الزمنية:"
 
-#: pretix/static/pretixpresale/js/ui/main.js:456
+#: pretix/static/pretixpresale/js/ui/main.js:482
 msgid "Your local time:"
 msgstr "التوقيت المحلي:"
 

src/pretix/locale/ca/LC_MESSAGES/djangojs.po

@@ -7,7 +7,7 @@ msgid ""
 msgstr ""
 "Project-Id-Version: PACKAGE VERSION\n"
 "Report-Msgid-Bugs-To: \n"
-"POT-Creation-Date: 2021-09-27 18:49+0000\n"
+"POT-Creation-Date: 2021-10-29 10:10+0000\n"
 "PO-Revision-Date: 2020-12-19 07:00+0000\n"
 "Last-Translator: albert <albert.serra.monner@gmail.com>\n"
 "Language-Team: Catalan <https://translate.pretix.eu/projects/pretix/pretix-"
@@ -194,6 +194,10 @@ msgstr ""
 msgid "Currently inside"
 msgstr ""
 
+#: pretix/static/lightbox/js/lightbox.js:96
+msgid "close"
+msgstr ""
+
 #: pretix/static/pretixbase/js/asynctask.js:43
 #: pretix/static/pretixbase/js/asynctask.js:119
 msgid ""
@@ -249,7 +253,7 @@ msgid ""
 "page and try again."
 msgstr ""
 
-#: pretix/static/pretixbase/js/asynctask.js:262
+#: pretix/static/pretixbase/js/asynctask.js:264
 #: pretix/static/pretixcontrol/js/ui/main.js:34
 msgid "Close message"
 msgstr ""
@@ -474,19 +478,25 @@ msgid_plural "({num} more dates)"
 msgstr[0] ""
 msgstr[1] ""
 
-#: pretix/static/pretixpresale/js/ui/cart.js:39
-msgid "The items in your cart are no longer reserved for you."
+#: pretix/static/pretixpresale/js/ui/cart.js:43
+#, fuzzy
+#| msgid "The items in your cart are no longer reserved for you."
+msgid ""
+"The items in your cart are no longer reserved for you. You can still "
+"complete your order as long as they’re available."
 msgstr "El contingut de la cistella ja no el teniu reservat."
 
-#: pretix/static/pretixpresale/js/ui/cart.js:41
+#: pretix/static/pretixpresale/js/ui/cart.js:45
 msgid "Cart expired"
 msgstr "Cistella expirada"
 
-#: pretix/static/pretixpresale/js/ui/cart.js:46
-msgid "The items in your cart are reserved for you for one minute."
-msgid_plural "The items in your cart are reserved for you for {num} minutes."
-msgstr[0] ""
-msgstr[1] ""
+#: pretix/static/pretixpresale/js/ui/cart.js:50
+#, fuzzy
+#| msgid "The items in your cart are no longer reserved for you."
+msgid "The items in your cart are reserved for you for one minute."
+msgid_plural "The items in your cart are reserved for you for {num} minutes."
+msgstr[0] "El contingut de la cistella ja no el teniu reservat."
+msgstr[1] "El contingut de la cistella ja no el teniu reservat."
 
 #: pretix/static/pretixpresale/js/ui/main.js:144
 msgid "The organizer keeps %(currency)s %(amount)s"
@@ -500,22 +510,22 @@ msgstr ""
 msgid "Please enter the amount the organizer can keep."
 msgstr ""
 
-#: pretix/static/pretixpresale/js/ui/main.js:309
+#: pretix/static/pretixpresale/js/ui/main.js:335
 msgid "Please enter a quantity for one of the ticket types."
 msgstr ""
 
-#: pretix/static/pretixpresale/js/ui/main.js:345
+#: pretix/static/pretixpresale/js/ui/main.js:371
 #, fuzzy
 #| msgid "Cart expired"
 msgid "required"
 msgstr "Cistella expirada"
 
-#: pretix/static/pretixpresale/js/ui/main.js:447
-#: pretix/static/pretixpresale/js/ui/main.js:465
+#: pretix/static/pretixpresale/js/ui/main.js:473
+#: pretix/static/pretixpresale/js/ui/main.js:491
 msgid "Time zone:"
 msgstr ""
 
-#: pretix/static/pretixpresale/js/ui/main.js:456
+#: pretix/static/pretixpresale/js/ui/main.js:482
 msgid "Your local time:"
 msgstr ""