eMail: Ratelimit exponential backup for Amazon SES

Currently translated at 100.0% (4658 of 4658 strings)

Translation: pretix/pretix
Translate-URL: https://translate.pretix.eu/projects/pretix/pretix/de_Informal/

powered by weblate

.github/workflows/tests.yml

@@ -55,7 +55,7 @@ jobs:
           restore-keys: |
             ${{ runner.os }}-pip-
       - name: Install system dependencies
-        run: sudo apt update && sudo apt install gettext mariadb-client
+        run: sudo apt update && sudo apt install gettext mariadb-client-10.3
       - name: Install Python dependencies
         run: pip3 install -e ".[dev]" mysqlclient psycopg2-binary
         working-directory: ./src

doc/admin/config.rst

@@ -65,6 +65,9 @@ Example::
     A comma-separated list of plugins that are not available even though they are installed.
     Defaults to an empty string.
 
+``plugins_show_meta``
+    Whether to show authors and versions of plugins, defaults to ``on``.
+
 ``auth_backends``
     A comma-separated list of available auth backends. Defaults to ``pretix.base.auth.NativeAuthBackend``.
 

doc/admin/installation/docker_smallscale.rst

@@ -36,9 +36,6 @@ Linux and firewalls, we recommend that you start with `ufw`_.
           SSL certificates can be obtained for free these days. We also *do not* provide support for HTTP-only
           installations except for evaluation purposes.
 
-.. warning:: We recommend **PostgreSQL**. If you go for MySQL, make sure you run **MySQL 5.7 or newer** or
-             **MariaDB 10.2.7 or newer**.
-
 .. warning:: By default, using `ufw` in conjunction will not have any effect. Please make sure to either bind the exposed
              ports of your docker container explicitly to 127.0.0.1 or configure docker to respect any set up firewall
              rules.
@@ -61,6 +58,9 @@ directory writable to the user that runs pretix inside the docker container::
 Database
 --------
 
+.. warning:: **Please use PostgreSQL for all new installations**. If you need to go for MySQL, make sure you run
+             **MySQL 5.7 or newer** or **MariaDB 10.2.7 or newer**.
+
 Next, we need a database and a database user. We can create these with any kind of database managing tool or directly on
 our database's shell. Please make sure that UTF8 is used as encoding for the best compatibility. You can check this with
 the following command::
@@ -91,6 +91,8 @@ When using MySQL, make sure you set the character set of the database to ``utf8m
 
     mysql > CREATE DATABASE pretix DEFAULT CHARACTER SET utf8mb4 DEFAULT COLLATE utf8mb4_unicode_ci;
 
+You will also need to make sure that ``sql_mode`` in your ``my.cnf`` file does **not** include ``ONLY_FULL_GROUP_BY``.
+
 Redis
 -----
 
@@ -106,6 +108,18 @@ Now restart redis-server::
 
     # systemctl restart redis-server
 
+In this setup, systemd will delete ``/var/run/redis`` on every redis restart, which will cause issues with pretix. To
+prevent this, you can execute::
+
+   # systemctl edit redis-server
+
+And insert the following::
+
+    [Service]
+    # Keep the directory around so that pretix.service in docker does not need to be
+    # restarted when redis is restarted.
+    RuntimeDirectoryPreserve=yes
+
 .. warning:: Setting the socket permissions to 777 is a possible security problem. If you have untrusted users on your
              system or have high security requirements, please don't do this and let redis listen to a TCP socket
              instead. We recommend the socket approach because the TCP socket in combination with docker's networking

doc/admin/installation/manual_smallscale.rst

@@ -34,9 +34,6 @@ Linux and firewalls, we recommend that you start with `ufw`_.
           SSL certificates can be obtained for free these days. We also *do not* provide support for HTTP-only
           installations except for evaluation purposes.
 
-.. warning:: We recommend **PostgreSQL**. If you go for MySQL, make sure you run **MySQL 5.7 or newer** or
-             **MariaDB 10.2.7 or newer**.
-
 Unix user
 ---------
 
@@ -50,6 +47,9 @@ In this guide, all code lines prepended with a ``#`` symbol are commands that yo
 Database
 --------
 
+.. warning:: **Please use PostgreSQL for all new installations**. If you need to go for MySQL, make sure you run
+             **MySQL 5.7 or newer** or **MariaDB 10.2.7 or newer**.
+
 Having the database server installed, we still need a database and a database user. We can create these with any kind
 of database managing tool or directly on our database's shell. Please make sure that UTF8 is used as encoding for the
 best compatibility. You can check this with the following command::
@@ -65,6 +65,8 @@ When using MySQL, make sure you set the character set of the database to ``utf8m
 
     mysql > CREATE DATABASE pretix DEFAULT CHARACTER SET utf8mb4 DEFAULT COLLATE utf8mb4_unicode_ci;
 
+You will also need to make sure that ``sql_mode`` in your ``my.cnf`` file does **not** include ``ONLY_FULL_GROUP_BY``.
+
 Package dependencies
 --------------------
 

doc/api/deviceauth.rst

@@ -99,7 +99,8 @@ following endpoint:
        "hardware_brand": "Samsung",
        "hardware_model": "Galaxy S",
        "software_brand": "pretixdroid",
-       "software_version": "4.1.0"
+       "software_version": "4.1.0",
+       "info": {"arbitrary": "data"}
    }
 
 You will receive a response equivalent to the response of your initialization request.

doc/api/fundamentals.rst

@@ -43,6 +43,8 @@ Possible permissions are:
 * Can view vouchers
 * Can change vouchers
 
+.. _`rest-compat`:
+
 Compatibility
 -------------
 
@@ -58,6 +60,7 @@ that your clients can deal with them properly:
 * Support of new HTTP methods for a given API endpoint
 * Support of new query parameters for a given API endpoint
 * New fields contained in API responses
+* Response body structure or message texts on failed requests (``4xx``, ``5xx`` response codes)
 
 We treat the following types of changes as *backwards-incompatible*:
 

doc/api/resources/index.rst

@@ -1,6 +1,11 @@
 Resources and endpoints
 =======================
 
+With a few exceptions, this only lists resources bundled in the pretix core modules.
+Additional endpoints are provided by pretix plugins. Some of them are documented
+at :ref:`plugin-docs`.
+
+
 .. toctree::
    :maxdepth: 2
 
@@ -33,4 +38,4 @@ Resources and endpoints
    exporters
    sendmail_rules
    billing_invoices
-   billing_var
+   billing_var

doc/api/resources/orders.rst

@@ -68,6 +68,7 @@ positions                             list of objects            List of order p
                                                                  non-canceled positions are included.
 fees                                  list of objects            List of fees included in the order total. By default, only
                                                                  non-canceled fees are included.
+├ id                                  integer                    Internal ID of the fee record
 ├ fee_type                            string                     Type of fee (currently ``payment``, ``passbook``,
                                                                  ``other``)
 ├ value                               money (string)             Fee amount
@@ -136,6 +137,10 @@ last_modified                         datetime                   Last modificati
 
    The ``subevent`` query parameters has been added.
 
+.. versionchanged:: 4.8
+
+   The ``order.fees.id`` attribute has been added.
+
 
 .. _order-position-resource:
 
@@ -735,6 +740,37 @@ Generating new secrets
    :statuscode 401: Authentication failure
    :statuscode 403: The requested organizer/event does not exist **or** you have no permission to update this order.
 
+.. http:post:: /api/v1/organizers/(organizer)/events/(event)/orderpositions/(id)/regenerate_secrets/
+
+   Triggers generation of a new ``secret`` attribute for a single order position.
+
+   **Example request**:
+
+   .. sourcecode:: http
+
+      POST /api/v1/organizers/bigevents/events/sampleconf/orderpositions/23/regenerate_secrets/ HTTP/1.1
+      Host: pretix.eu
+      Accept: application/json, text/javascript
+
+   **Example response**:
+
+   .. sourcecode:: http
+
+      HTTP/1.1 200 OK
+      Vary: Accept
+      Content-Type: application/json
+
+      (Full order position resource, see above.)
+
+   :param organizer: The ``slug`` field of the organizer of the event
+   :param event: The ``slug`` field of the event
+   :param code: The ``id`` field of the order position to update
+
+   :statuscode 200: no error
+   :statuscode 400: The order position could not be updated due to invalid submitted data.
+   :statuscode 401: Authentication failure
+   :statuscode 403: The requested organizer/event does not exist **or** you have no permission to update this order position.
+
 Deleting orders
 ---------------
 
@@ -839,6 +875,7 @@ Creating orders
    * ``comment`` (optional)
    * ``custom_followup_at`` (optional)
    * ``checkin_attention`` (optional)
+   * ``require_approval`` (optional)
    * ``invoice_address`` (optional)
 
       * ``company``
@@ -898,8 +935,9 @@ Creating orders
 
    * ``force`` (optional). If set to ``true``, quotas will be ignored.
    * ``send_email`` (optional). If set to ``true``, the same emails will be sent as for a regular order, regardless of
-     whether these emails are enabled for certain sales channels. Defaults to
-     ``false``. Used to be ``send_mail`` before pretix 3.14.
+     whether these emails are enabled for certain sales channels. If set to ``null``, behavior will be controlled by pretix'
+     settings based on the sales channels (added in pretix 4.7). Defaults to ``false``.
+     Used to be ``send_mail`` before pretix 3.14.
 
    If you want to use add-on products, you need to set the ``positionid`` fields of all positions manually
    to incrementing integers starting with ``1``. Then, you can reference one of these
@@ -1044,6 +1082,9 @@ Order state operations
    will instead stay paid, but all positions will be removed (or marked as canceled) and replaced by the cancellation
    fee as the only component of the order.
 
+   You can control whether the customer is notified through ``send_email`` (defaults to ``true``).
+   You can pass a ``comment`` that can be visible to the user if it is used in the email template.
+
    **Example request**:
 
    .. sourcecode:: http
@@ -1055,6 +1096,7 @@ Order state operations
 
       {
           "send_email": true,
+          "comment": "Event was canceled.",
           "cancellation_fee": null
       }
 
@@ -1640,6 +1682,8 @@ Order position ticket download
    :statuscode 409: The file is not yet ready and will now be prepared. Retry the request after waiting for a few
                     seconds.
 
+.. _rest-orderpositions-manipulate:
+
 Manipulating individual positions
 ---------------------------------
 
@@ -1647,6 +1691,11 @@ Manipulating individual positions
 
    The ``PATCH`` method has been added for individual positions.
 
+.. versionchanged:: 4.8
+
+   The ``PATCH`` method now supports changing items, variations, subevents, seats, prices, and tax rules.
+   The ``POST`` endpoint to add individual positions has been added.
+
 .. http:patch:: /api/v1/organizers/(organizer)/events/(event)/orderpositions/(id)/
 
    Updates specific fields on an order position. Currently, only the following fields are supported:
@@ -1673,6 +1722,21 @@ Manipulating individual positions
      and ``option_identifiers`` will be ignored. As a special case, you can submit the magic value
      ``"file:keep"`` as the answer to a file question to keep the current value without re-uploading it.
 
+   * ``item``
+
+   * ``variation``
+
+   * ``subevent``
+
+   * ``seat`` (specified as a string mapping to a ``string_guid``)
+
+   * ``price``
+
+   * ``tax_rule``
+
+   Changing parameters such as ``item`` or ``price`` will **not** automatically trigger creation of a new invoice,
+   you need to take care of that yourself.
+
    **Example request**:
 
    .. sourcecode:: http
@@ -1694,7 +1758,7 @@ Manipulating individual positions
       Vary: Accept
       Content-Type: application/json
 
-      (Full order resource, see above.)
+      (Full order position resource, see above.)
 
    :param organizer: The ``slug`` field of the organizer of the event
    :param event: The ``slug`` field of the event
@@ -1705,9 +1769,83 @@ Manipulating individual positions
    :statuscode 401: Authentication failure
    :statuscode 403: The requested organizer/event does not exist **or** you have no permission to update this order.
 
+.. http:post:: /api/v1/organizers/(organizer)/events/(event)/orderpositions/
+
+   Adds a new position to an order. Currently, only the following fields are supported:
+
+   * ``order`` (mandatory, specified as a string mapping to a ``code``)
+
+   * ``addon_to`` (optional, specified as an integer mapping to the ``positionid`` of the parent position)
+
+   * ``item`` (mandatory)
+
+   * ``variation`` (mandatory depending on item)
+
+   * ``subevent`` (mandatory depending on event)
+
+   * ``seat`` (specified as a string mapping to a ``string_guid``, mandatory depending on event and item)
+
+   * ``price`` (default price will be used if unset)
+
+   * ``attendee_email``
+
+   * ``attendee_name_parts`` or ``attendee_name``
+
+   * ``company``
+
+   * ``street``
+
+   * ``zipcode``
+
+   * ``city``
+
+   * ``country``
+
+   * ``state``
+
+   * ``answers``: Validation is handled the same way as when creating orders through the API. You are therefore
+     expected to provide ``question``, ``answer``, and possibly ``options``. ``question_identifier``
+     and ``option_identifiers`` will be ignored. As a special case, you can submit the magic value
+     ``"file:keep"`` as the answer to a file question to keep the current value without re-uploading it.
+
+   This will **not** automatically trigger creation of a new invoice, you need to take care of that yourself.
+
+   **Example request**:
+
+   .. sourcecode:: http
+
+      POST /api/v1/organizers/bigevents/events/sampleconf/orderpositions/ HTTP/1.1
+      Host: pretix.eu
+      Accept: application/json, text/javascript
+      Content-Type: application/json
+
+      {
+        "order": "ABC12",
+        "item": 5,
+        "addon_to": 1
+      }
+
+   **Example response**:
+
+   .. sourcecode:: http
+
+      HTTP/1.1 201 Created
+      Vary: Accept
+      Content-Type: application/json
+
+      (Full order position resource, see above.)
+
+   :param organizer: The ``slug`` field of the organizer of the event
+   :param event: The ``slug`` field of the event
+
+   :statuscode 200: no error
+   :statuscode 400: The position could not be created due to invalid submitted data.
+   :statuscode 401: Authentication failure
+   :statuscode 403: The requested organizer/event does not exist **or** you have no permission to create this position.
+
 .. http:delete:: /api/v1/organizers/(organizer)/events/(event)/orderpositions/(id)/
 
-   Deletes an order position, identified by its internal ID.
+   Cancels an order position, identified by its internal ID.
 
    **Example request**:
 
@@ -1733,6 +1871,128 @@ Manipulating individual positions
    :statuscode 403: The requested organizer/event does not exist **or** you have no permission to view this resource.
    :statuscode 404: The requested order position does not exist.
 
+Changing order contents
+-----------------------
+
+While you can :ref:`change positions individually <rest-orderpositions-manipulate>` sometimes it is necessary to make
+multiple changes to an order at once within one transaction. This makes it possible to e.g. swap the seats of two
+attendees in an order without running into conflicts. This interface also offers some possibilities not available
+otherwise, such as splitting an order or changing fees.
+
+.. versionchanged:: 4.8
+
+   This endpoint has been added to the system.
+
+.. http:post:: /api/v1/organizers/(organizer)/events/(event)/orders/(code)/change/
+
+   Performs a change operation on an order. You can supply the following fields:
+
+   * ``patch_positions``: A list of objects with the two keys ``position`` specifying an order position ID and
+     ``body`` specifying the desired changed values of the position (``item``, ``variation``, ``subevent``, ``seat``,
+     ``price``, ``tax_rule``).
+
+   * ``cancel_positions``: A list of objects with the single key ``position`` specifying an order position ID.
+
+   * ``split_positions``: A list of objects with the single key ``position`` specifying an order position ID.
+
+   * ``create_positions``: A list of objects describing new order positions with the same fields supported as when
+     creating them individually through the ``POST …/orderpositions/`` endpoint.
+
+   * ``patch_fees``: A list of objects with the two keys ``fee`` specifying an order fee ID and
+     ``body`` specifying the desired changed values of the position (``value``).
+
+   * ``cancel_fees``: A list of objects with the single key ``fee`` specifying an order fee ID.
+
+   * ``recalculate_taxes``: If set to ``"keep_net"``, all taxes will be recalculated based on the tax rule and invoice
+     address, the net price will be kept. If set to ``"keep_gross"``, the gross price will be kept. If set to ``null``
+     (the default) the taxes are not recalculated.
+
+   * ``send_email``: If set to ``true``, the customer will be notified about the change. Defaults to ``false``.
+
+   * ``reissue_invoice``: If set to ``true`` and an invoice exists for the order, it will be canceled and a new invoice
+     will be issued. Defaults to ``true``.
+
+   **Example request**:
+
+   .. sourcecode:: http
+
+      POST /api/v1/organizers/bigevents/events/sampleconf/orders/ABC12/ HTTP/1.1
+      Host: pretix.eu
+      Accept: application/json, text/javascript
+      Content-Type: application/json
+
+      {
+        "cancel_positions": [
+          {
+            "position": 12373
+          }
+        ],
+        "patch_positions": [
+          {
+            "position": 12374,
+            "body": {
+              "item": 12,
+              "variation": None,
+              "subevent": 562,
+              "seat": "seat-guid-2",
+              "price": "99.99",
+              "tax_rule": 15
+            }
+          }
+        ],
+        "split_positions": [
+          {
+            "position": 12375
+          }
+        ],
+        "create_positions": [
+          {
+            "item": 12,
+            "variation": None,
+            "subevent": 562,
+            "seat": "seat-guid-2",
+            "price": "99.99",
+            "addon_to": 12374,
+            "attendee_name": "Peter",
+          }
+        ],
+        "cancel_fees": [
+          {
+            "fee": 49
+          }
+        ],
+        "change_fees": [
+          {
+            "fee": 51,
+            "body": {
+              "value": "12.00"
+            }
+          }
+        ],
+        "reissue_invoice": true,
+        "send_email": true,
+        "recalculate_taxes": "keep_gross"
+      }
+
+   **Example response**:
+
+   .. sourcecode:: http
+
+      HTTP/1.1 200 OK
+      Vary: Accept
+      Content-Type: application/json
+
+      (Full order position resource, see above.)
+
+   :param organizer: The ``slug`` field of the organizer of the event
+   :param event: The ``slug`` field of the event
+   :param code: The ``code`` field of the order to update
+
+   :statuscode 200: no error
+   :statuscode 400: The order could not be updated due to invalid submitted data.
+   :statuscode 401: Authentication failure
+   :statuscode 403: The requested organizer/event does not exist **or** you have no permission to update this order.
+
 
 Order payment endpoints
 -----------------------

doc/development/api/auth.rst

@@ -20,20 +20,31 @@ Basically, three pre-defined flows are supported:
 * Authentication mechanisms that rely on **redirection**, e.g. to an OAuth provider. These can be implemented by
   supplying a ``authentication_url`` method and implementing a custom return view.
 
-Authentication backends are *not* collected through a signal. Instead, they must explicitly be set through the
-``auth_backends`` directive in the ``pretix.cfg`` :ref:`configuration file <config>`.
+For security reasons, authentication backends are *not* automatically discovered through a signal. Instead, they must
+explicitly be set through the ``auth_backends`` directive in the ``pretix.cfg`` :ref:`configuration file <config>`.
 
-In each of these methods (``form_authenticate``, ``request_authenticate`` or your custom view) you are supposed to
-either get an existing :py:class:`pretix.base.models.User` object from the database or create a new one. There are a
-few rules you need to follow:
+In each of these methods (``form_authenticate``, ``request_authenticate``, or your custom view) you are supposed to
+use ``User.objects.get_or_create_for_backend`` to get a :py:class:`pretix.base.models.User` object from the database
+or create a new one.
 
-* You **MUST** only return users with the ``auth_backend`` attribute set to the ``identifier`` value of your backend.
+There are a few rules you need to follow:
 
-* You **MUST** create new users with the ``auth_backend`` attribute set to the ``identifier`` value of your backend.
+* You **MUST** have some kind of identifier for a user that is globally unique and **SHOULD** never change, even if the
+  user's name or email address changes. This could e.g. be the ID of the user in an external database. The identifier
+  must not be longer than 190 characters. If you worry your backend might generated longer identifiers, consider
+  using a hash function to trim them to a constant length.
+
+* You **SHOULD** not allow users created by other authentication backends to log in through your code, and you **MUST**
+  only create, modify or return users with ``auth_backend`` set to your backend.
 
 * Every user object **MUST** have an email address. Email addresses are globally unique. If the email address is
   already registered to a user who signs in through a different backend, you **SHOULD** refuse the login.
 
+``User.objects.get_or_create_for_backend`` will follow these rules for you automatically. It works like this:
+
+.. autoclass:: pretix.base.models.auth.UserManager
+   :members: get_or_create_for_backend
+
 The backend interface
 ---------------------
 
@@ -59,6 +70,7 @@ The backend interface
 
    .. automethod:: authentication_url
 
+
 Logging users in
 ----------------
 
@@ -68,3 +80,45 @@ recommend that you use the following utility method to correctly set session val
 authentication (if activated):
 
 .. autofunction:: pretix.control.views.auth.process_login
+
+A custom view that is called after a redirect from an external identity provider could look like this::
+
+   from django.contrib import messages
+   from django.shortcuts import redirect
+   from django.urls import reverse
+
+   from pretix.base.models import User
+   from pretix.base.models.auth import EmailAddressTakenError
+   from pretix.control.views.auth import process_login
+
+
+   def return_view(request):
+       # Verify validity of login with the external provider's API
+       api_response = my_verify_login_function(
+           code=request.GET.get('code')
+       )
+
+       try:
+           u = User.objects.get_or_create_for_backend(
+               'my_backend_name',
+               api_response['userid'],
+               api_response['email'],
+               set_always={
+                   'fullname': '{} {}'.format(
+                       api_response.get('given_name', ''),
+                       api_response.get('family_name', ''),
+                   ),
+               },
+               set_on_creation={
+                   'locale': api_response.get('locale').lower()[:2],
+                   'timezone': api_response.get('zoneinfo', 'UTC'),
+               }
+           )
+       except EmailAddressTakenError:
+           messages.error(
+               request, _('We cannot create your user account as a user account in this system '
+                          'already exists with the same email address.')
+           )
+           return redirect(reverse('control:auth.login'))
+       else:
+           return process_login(request, u, keep_logged_in=False)

doc/development/api/plugins.rst

@@ -45,13 +45,17 @@ Attribute          Type                 Description
 name               string               The human-readable name of your plugin
 author             string               Your name
 version            string               A human-readable version code of your plugin
-description        string               A more verbose description of what your plugin does.
+description        string               A more verbose description of what your plugin does. May contain HTML.
 category           string               Category of a plugin. Either one of ``"FEATURE"``, ``"PAYMENT"``,
                                         ``"INTEGRATION"``, ``"CUSTOMIZATION"``, ``"FORMAT"``, or ``"API"``,
                                         or any other string.
+picture            string (optional)    Path to a picture resolvable through the static file system.
+featured           boolean (optional)   ``False`` by default, can promote a plugin if it's something many users will want, use carefully.
 visible            boolean (optional)   ``True`` by default, can hide a plugin so it cannot be normally activated.
 restricted         boolean (optional)   ``False`` by default, restricts a plugin such that it can only be enabled
                                         for an event by system administrators / superusers.
+experimental       boolean (optional)   ``False`` by default, marks a plugin as an experimental feature in the plugins list.
+picture            string (optional)    Path to a picture resolvable through the static file system.
 compatibility      string               Specifier for compatible pretix versions.
 ================== ==================== ===========================================================
 
@@ -74,8 +78,10 @@ A working example would be:
             name = _("PayPal")
             author = _("the pretix team")
             version = '1.0.0'
-            category = 'PAYMENT
+            category = 'PAYMENT'
+            picture = 'pretix_paypal/paypal_logo.svg'
             visible = True
+            featured = False
             restricted = False
             description = _("This plugin allows you to receive payments via PayPal")
             compatibility = "pretix>=2.7.0"
@@ -92,6 +98,7 @@ those will be displayed but not block the plugin execution.
 
 The ``AppConfig`` class may implement a method ``is_available(event)`` that checks if a plugin
 is available for a specific event. If not, it will not be shown in the plugin list of that event.
+You should not define ``is_available`` and ``restricted`` on the same plugin.
 
 Plugin registration
 -------------------

doc/plugins/digital.rst

@@ -61,7 +61,7 @@ Variable                            Description
 ``attendee_city``                   City of the ticket holder's address (or empty)
 ``attendee_country``                Country code of the ticket holder's address (or empty)
 ``attendee_state``                  State of the ticket holder's address (or empty)
-``answer[XYZ]``                     Answer to the custom question with identifier ``XYZ``
+``answers[XYZ]``                    Answer to the custom question with identifier ``XYZ``
 ``invoice_name``                    Full name of the invoice address (or empty)
 ``invoice_name_*``                  Name parts of the invoice address, depending on configuration, e.g. ``invoice_name_given_name`` or ``invoice_name_family_name``
 ``invoice_company``                 Company of the invoice address (or empty)

doc/plugins/exhibitors.rst

@@ -0,0 +1,630 @@
+Exhibitors
+==========
+
+The exhibitors plugin allows to manage exhibitors at your trade show or conference. After signing up your exhibitors
+in the system, you can assign vouchers to exhibitors and give them access to the data of these vouchers. The exhibitors
+module is also the basis of the pretixLEAD lead scanning application.
+
+.. note:: On pretix Hosted, using the lead scanning feature of the exhibitors plugin can add additional costs
+          depending on your contract.
+
+The plugin exposes two APIs. One (REST API) is intended for bulk-data operations from the admin side, and one
+(App API) that is used by the pretixLEAD app.
+
+REST API
+---------
+
+The REST API for exhibitors requires the usual :ref:`rest-auth`.
+
+Resources
+"""""""""
+
+The exhibitors plugin provides a HTTP API that allows you to create new exhibitors.
+
+The exhibitors resource contains the following public fields:
+
+.. rst-class:: rest-resource-table
+
+===================================== ========================== =======================================================
+Field                                 Type                       Description
+===================================== ========================== =======================================================
+id                                    integer                    Internal exhibitor ID in pretix
+name                                  string                     Exhibitor name
+internal_id                           string                     Can be used for the ID in your exhibition system, your customer ID, etc. Can be ``null``. Maximum 255 characters.
+contact_name                          string                     Contact person (or ``null``)
+contact_name_parts                    object of strings          Decomposition of contact name (i.e. given name, family name)
+contact_email                         string                     Contact person email address (or ``null``)
+booth                                 string                     Booth number (or ``null``). Maximum 100 characters.
+locale                                string                     Locale for communication with the exhibitor (or ``null``).
+access_code                           string                     Access code for the exhibitor to access their data or use the lead scanning app (read-only).
+allow_lead_scanning                   boolean                    Enables lead scanning app
+allow_lead_access                     boolean                    Enables access to data gathered by the lead scanning app
+allow_voucher_access                  boolean                    Enables access to data gathered by exhibitor vouchers
+comment                               string                     Internal comment, not shown to exhibitor
+===================================== ========================== =======================================================
+
+You can also access the scanned leads through the API which contains the following public fields:
+
+.. rst-class:: rest-resource-table
+
+===================================== ========================== =======================================================
+Field                                 Type                       Description
+===================================== ========================== =======================================================
+attendee_order                        string                     Order code of the order the scanned attendee belongs to
+attendee_positionid                   integer                    ``positionid`` if the attendee within the order specified by ``attendee_order``
+rating                                integer                    A rating of 0 to 5 stars (or ``null``)
+notes                                 string                     A note taken by the exhibitor after scanning
+tags                                  list of strings            Additional tags selected by the exhibitor
+first_upload                          datetime                   Date and time of the first upload of this lead
+data                                  list of objects            Attendee data set that may be shown to the exhibitor based o
+                                                                 the event's configuration. Each entry contains the fields ``id``,
+                                                                 ``label``, ``value``, and ``details``. ``details`` is usually empty
+                                                                 except in a few cases where it contains an additional list of objects
+                                                                 with ``value`` and ``label`` keys (e.g. splitting of names).
+device_name                           string                     User-defined name for the device used for scanning (or ``null``).
+===================================== ========================== =======================================================
+
+Endpoints
+"""""""""
+
+.. http:get:: /api/v1/organizers/(organizer)/events/(event)/exhibitors/
+
+   Returns a list of all exhibitors configured for an event.
+
+   **Example request**:
+
+   .. sourcecode:: http
+
+      GET /api/v1/organizers/bigevents/events/sampleconf/exhibitors/ HTTP/1.1
+      Host: pretix.eu
+      Accept: application/json, text/javascript
+
+   **Example response**:
+
+   .. sourcecode:: http
+
+      HTTP/1.1 200 OK
+      Vary: Accept
+      Content-Type: application/json
+
+      {
+        "count": 1,
+        "next": null,
+        "previous": null,
+        "results": [
+          {
+            "id": 1,
+            "name": "Aperture Science",
+            "internal_id": null,
+            "contact_name": "Dr Cave Johnson",
+            "contact_name_parts": {
+                "_scheme": "salutation_title_given_family",
+                "family_name": "Johnson",
+                "given_name": "Cave",
+                "salutation": "",
+                "title": "Dr"
+            },
+            "contact_email": "johnson@as.example.org",
+            "booth": "A2",
+            "locale": "de",
+            "access_code": "VKHZ2FU8",
+            "allow_lead_scanning": true,
+            "allow_lead_access": true,
+            "allow_voucher_access": true,
+            "comment": ""
+          }
+        ]
+      }
+
+   :query page: The page number in case of a multi-page result set, default is 1
+   :param organizer: The ``slug`` field of a valid organizer
+   :param event: The ``slug`` field of the event to fetch
+   :statuscode 200: no error
+   :statuscode 401: Authentication failure
+   :statuscode 403: The requested organizer or event does not exist **or** you have no permission to view it.
+
+.. http:get:: /api/v1/organizers/(organizer)/events/(event)/exhibitors/(id)/
+
+   Returns information on one exhibitor, identified by its ID.
+
+   **Example request**:
+
+   .. sourcecode:: http
+
+      GET /api/v1/organizers/bigevents/events/sampleconf/exhibitors/1/ HTTP/1.1
+      Host: pretix.eu
+      Accept: application/json, text/javascript
+
+   **Example response**:
+
+   .. sourcecode:: http
+
+      HTTP/1.1 200 OK
+      Vary: Accept
+      Content-Type: application/json
+
+      {
+        "id": 1,
+        "name": "Aperture Science",
+        "internal_id": null,
+        "contact_name": "Dr Cave Johnson",
+        "contact_name_parts": {
+            "_scheme": "salutation_title_given_family",
+            "family_name": "Johnson",
+            "given_name": "Cave",
+            "salutation": "",
+            "title": "Dr"
+        },
+        "contact_email": "johnson@as.example.org",
+        "booth": "A2",
+        "locale": "de",
+        "access_code": "VKHZ2FU8",
+        "allow_lead_scanning": true,
+        "allow_lead_access": true,
+        "allow_voucher_access": true,
+        "comment": ""
+      }
+
+   :param organizer: The ``slug`` field of the organizer to fetch
+   :param event: The ``slug`` field of the event to fetch
+   :param id: The ``id`` field of the exhibitor to fetch
+   :statuscode 200: no error
+   :statuscode 401: Authentication failure
+   :statuscode 403: The requested organizer/event/exhibitor does not exist **or** you have no permission to view it.
+
+.. http:get:: /api/v1/organizers/(organizer)/events/(event)/exhibitors/(id)/leads/
+
+   Returns a list of all scanned leads of an exhibitor.
+
+   **Example request**:
+
+   .. sourcecode:: http
+
+      GET /api/v1/organizers/bigevents/events/sampleconf/exhibitors/1/leads/ HTTP/1.1
+      Host: pretix.eu
+      Accept: application/json, text/javascript
+
+   **Example response**:
+
+   .. sourcecode:: http
+
+      HTTP/1.1 200 OK
+      Vary: Accept
+      Content-Type: application/json
+
+      {
+        "count": 1,
+        "next": null,
+        "previous": null,
+        "results": [
+          {
+            "attendee_order": "T0E7E",
+            "attendee_positionid": 1,
+            "rating": 1,
+            "notes": "",
+            "tags": [],
+            "first_upload": "2021-07-06T11:03:31.414491+01:00",
+            "data": [
+              {
+                "id": "attendee_name",
+                "label": "Attendee name",
+                "value": "Peter Miller",
+                "details": [
+                  {"label": "Given name", "value": "Peter"},
+                  {"label": "Family name", "value": "Miller"},
+                ]
+              }
+            ]
+          }
+        ]
+      }
+
+   :query page: The page number in case of a multi-page result set, default is 1
+   :param organizer: The ``slug`` field of a valid organizer
+   :param event: The ``slug`` field of the event to fetch
+   :param id: The ``id`` field of the exhibitor to fetch
+   :statuscode 200: no error
+   :statuscode 401: Authentication failure
+   :statuscode 403: The requested organizer or event or exhibitor does not exist **or** you have no permission to view it.
+
+.. http:post:: /api/v1/organizers/(organizer)/events/(event)/exhibitors/
+
+   Create a new exhibitor.
+
+   **Example request**:
+
+   .. sourcecode:: http
+
+      POST /api/v1/organizers/bigevents/events/sampleconf/exhibitors/ HTTP/1.1
+      Host: pretix.eu
+      Accept: application/json, text/javascript
+      Content-Type: application/json
+      Content-Length: 166
+
+      {
+        "name": "Aperture Science",
+        "internal_id": null,
+        "contact_name_parts": {
+            "_scheme": "salutation_title_given_family",
+            "family_name": "Johnson",
+            "given_name": "Cave",
+            "salutation": "",
+            "title": "Dr"
+        },
+        "contact_email": "johnson@as.example.org",
+        "booth": "A2",
+        "locale": "de",
+        "access_code": "VKHZ2FU8",
+        "allow_lead_scanning": true,
+        "allow_lead_access": true,
+        "allow_voucher_access": true,
+        "comment": ""
+      }
+
+   **Example response**:
+
+   .. sourcecode:: http
+
+      HTTP/1.1 201 Created
+      Vary: Accept
+      Content-Type: application/json
+
+      {
+        "id": 1,
+        "name": "Aperture Science",
+        "internal_id": null,
+        "contact_name": "Dr Cave Johnson",
+        "contact_name_parts": {
+            "_scheme": "salutation_title_given_family",
+            "family_name": "Johnson",
+            "given_name": "Cave",
+            "salutation": "",
+            "title": "Dr"
+        },
+        "contact_email": "johnson@as.example.org",
+        "booth": "A2",
+        "locale": "de",
+        "access_code": "VKHZ2FU8",
+        "allow_lead_scanning": true,
+        "allow_lead_access": true,
+        "allow_voucher_access": true,
+        "comment": ""
+      }
+
+   :param organizer: The ``slug`` field of the organizer to create new exhibitor for
+   :param event: The ``slug`` field of the event to create new exhibitor for
+   :statuscode 201: no error
+   :statuscode 400: The exhibitor could not be created due to invalid submitted data.
+   :statuscode 401: Authentication failure
+   :statuscode 403: The requested organizer/event does not exist **or** you have no permission to create exhibitors.
+
+
+.. http:patch:: /api/v1/organizers/(organizer)/events/(event)/exhibitors/(id)/
+
+   Update an exhibitor. You can also use ``PUT`` instead of ``PATCH``. With ``PUT``, you have to provide all fields of
+   the resource, other fields will be reset to default. With ``PATCH``, you only need to provide the fields that you
+   want to change.
+
+   **Example request**:
+
+   .. sourcecode:: http
+
+      PATCH /api/v1/organizers/bigevents/events/sampleconf/digitalcontents/1/ HTTP/1.1
+      Host: pretix.eu
+      Accept: application/json, text/javascript
+      Content-Type: application/json
+      Content-Length: 34
+
+      {
+        "internal_id": "ABC"
+      }
+
+   **Example response**:
+
+   .. sourcecode:: http
+
+      HTTP/1.1 200 OK
+      Vary: Accept
+      Content-Type: text/javascript
+
+      {
+        "id": 1,
+        "name": "Aperture Science",
+        "internal_id": "ABC",
+        "contact_name": "Dr Cave Johnson",
+        "contact_name_parts": {
+            "_scheme": "salutation_title_given_family",
+            "family_name": "Johnson",
+            "given_name": "Cave",
+            "salutation": "",
+            "title": "Dr"
+        },
+        "contact_email": "johnson@as.example.org",
+        "booth": "A2",
+        "locale": "de",
+        "access_code": "VKHZ2FU8",
+        "allow_lead_scanning": true,
+        "allow_lead_access": true,
+        "allow_voucher_access": true,
+        "comment": ""
+      }
+
+   :param organizer: The ``slug`` field of the organizer to modify
+   :param event: The ``slug`` field of the event to modify
+   :param id: The ``id`` field of the exhibitor to modify
+   :statuscode 200: no error
+   :statuscode 400: The exhibitor could not be modified due to invalid submitted data.
+   :statuscode 401: Authentication failure
+   :statuscode 403: The requested organizer/event/exhibitor does not exist **or** you have no permission to change it.
+
+
+.. http:delete:: /api/v1/organizers/(organizer)/events/(event)/exhibitors/(id)/
+
+   Delete an exhibitor.
+
+   .. warning:: This deletes all lead scan data and removes all connections to vouchers (the vouchers are not deleted).
+
+   **Example request**:
+
+   .. sourcecode:: http
+
+      DELETE /api/v1/organizers/bigevents/events/sampleconf/exhibitors/1/ HTTP/1.1
+      Host: pretix.eu
+      Accept: application/json, text/javascript
+
+   **Example response**:
+
+   .. sourcecode:: http
+
+      HTTP/1.1 204 No Content
+      Vary: Accept
+
+   :param organizer: The ``slug`` field of the organizer to modify
+   :param event: The ``slug`` field of the event to modify
+   :param id: The ``id`` field of the exhibitor to delete
+   :statuscode 204: no error
+   :statuscode 401: Authentication failure
+   :statuscode 403: The requested organizer/event/exhibitor does not exist **or** you have no permission to change it
+
+
+App API
+-------
+
+The App API is used for communication between the pretixLEAD app and the pretix server.
+
+.. warning:: We consider this an internal API, it is not intended for external use. You may still use it, but
+             our :ref:`compatibility commitment <rest-compat>` does not apply.
+
+Authentication
+""""""""""""""
+
+Every exhibitor has an "access code", usually consisting of 8 alphanumeric uppercase characters.
+This access code is communicated to event exhibitors by the event organizers, so this is also what
+exhibitors should enter into a login screen.
+
+All API requests need to contain this access code as a header like this::
+
+    Authorization: Exhibitor ABCDE123
+
+Exhibitor profile
+"""""""""""""""""
+
+Upon login and in regular intervals after that, the API should fetch the exhibitors profile.
+This serves two purposes:
+
+* Checking if the authorization code is actually valid
+
+* Obtaining information that can be shown in the app
+
+The resource consists of the following fields:
+
+.. rst-class:: rest-resource-table
+
+===================================== ========================== =======================================================
+Field                                 Type                       Description
+===================================== ========================== =======================================================
+name                                  string                     Exhibitor name
+booth                                 string                     Booth number (or ``null``)
+event                                 object                     Object describing the event
+├ name                                multi-lingual string       Event name
+├ imprint_url                         string                     URL to legal notice page. If not ``null``, a button in the app should link to this page.
+├ privacy_url                         string                     URL to privacy notice page. If not ``null``, a button in the app should link to this page.
+├ help_url                            string                     URL to help page. If not ``null``, a button in the app should link to this page.
+├ logo_url                            string                     URL to event logo. If not ``null``, this logo may be shown in the app.
+├ slug                                string                     Event short form
+└ organizer                           string                     Organizer short form
+notes                                 boolean                    Specifies whether the exhibitor is allowed to take notes on leads
+tags                                  list of strings            List of tags the exhibitor can assign to their leads
+scan_types                            list of objects            Only used for a special case, fixed value that external API consumers should ignore
+===================================== ========================== =======================================================
+
+.. http:get:: /exhibitors/api/v1/profile
+
+   **Example request:**
+
+   .. sourcecode:: http
+
+    GET /exhibitors/api/v1/profile HTTP/1.1
+    Authorization: Exhibitor ABCDE123
+    Accept: application/json, text/javascript
+
+   **Example response:**
+
+   .. sourcecode:: http
+
+    HTTP/1.1 200 OK
+    Vary: Accept
+    Content-Type: application/json
+
+    {
+      "name": "Aperture Science",
+      "booth": "A2",
+      "event": {
+        "name": {"en": "Sample conference", "de": "Beispielkonferenz"},
+        "slug": "bigevents",
+        "imprint_url": null,
+        "privacy_url": null,
+        "help_url": null,
+        "logo_url": null,
+        "organizer": "sampleconf"
+      },
+      "notes": true,
+      "tags": ["foo", "bar"],
+      "scan_types": [
+        {
+          "key": "lead",
+          "label": "Lead Scanning"
+        }
+      ]
+    }
+
+   :statuscode 200: no error
+   :statuscode 401: Invalid authentication code
+
+Submitting a lead
+"""""""""""""""""
+
+After a ticket/badge is scanned, it should immediately be submitted to the server
+so the scan is stored and information about the person can be shown in the app. The same
+code can be submitted multiple times, so it's no problem to just submit it again after the
+exhibitor set a note or a rating (0-5) inside the app.
+
+On the request, you should set the following properties:
+
+* ``code`` with the scanned barcode
+* ``notes`` with the exhibitor's notes
+* ``scanned`` with the date and time of the actual scan (not the time of the upload)
+* ``scan_type`` set to ``lead`` statically
+* ``tags`` with the list of selected tags
+* ``rating`` with the rating assigned by the exhibitor
+* ``device_name`` with a user-specified name of the device used for scanning (max. 190 characters), or ``null``
+
+If you submit ``tags`` and ``rating`` to be ``null`` and ``notes`` to be ``""``, the server
+responds with the previously saved information and will not delete that information. If you
+supply other values, the information saved on the server will be overridden.
+
+The response will also contain ``tags``, ``rating``, and ``notes``. Additionally,
+it will include ``attendee`` with a list of ``fields`` that can be shown to the
+user. Each field has an internal ``id``, a human-readable ``label``, and a ``value`` (all strings).
+
+Note that the ``fields`` array can contain any number of dynamic keys!
+Depending on the exhibitors permission and event configuration this might be empty,
+or contain lots of details. The app should dynamically show these values (read-only)
+with the labels sent by the server.
+
+The request for this looks like this:
+
+.. http:post:: /exhibitors/api/v1/leads/
+
+   **Example request:**
+
+   .. sourcecode:: http
+
+    POST /exhibitors/api/v1/leads/ HTTP/1.1
+    Authorization: Exhibitor ABCDE123
+    Accept: application/json, text/javascript
+    Content-Type: application/json
+
+    {
+      "code": "qrcodecontent",
+      "notes": "Great customer, wants our newsletter",
+      "scanned": "2020-10-18T12:24:23.000+00:00",
+      "scan_type": "lead",
+      "tags": ["foo"],
+      "rating": 4,
+      "device_name": "DEV1"
+    }
+
+   **Example response:**
+
+   .. sourcecode:: http
+
+    HTTP/1.1 201 Created
+    Vary: Accept
+    Content-Type: application/json
+
+    {
+      "attendee": {
+        "fields": [
+          {
+            "id": "attendee_name",
+            "label": "Name",
+            "value": "Jon Doe",
+            "details": [
+              {"label": "Given name", "value": "John"},
+              {"label": "Family name", "value": "Doe"},
+            ]
+          },
+          {
+            "id": "attendee_email",
+            "label": "Email",
+            "value": "test@example.com",
+            "details": []
+          }
+         ]
+        },
+        "rating": 4,
+        "tags": ["foo"],
+        "notes": "Great customer, wants our newsletter"
+    }
+
+   :statuscode 200: No error, leads was not scanned for the first time
+   :statuscode 201: No error, leads was scanned for the first time
+   :statuscode 400: Invalid data submitted
+   :statuscode 401: Invalid authentication code
+
+You can also fetch existing leads (if you are authorized to do so):
+
+.. http:get:: /exhibitors/api/v1/leads/
+
+   **Example request:**
+
+   .. sourcecode:: http
+
+    GET /exhibitors/api/v1/leads/ HTTP/1.1
+    Authorization: Exhibitor ABCDE123
+    Accept: application/json, text/javascript
+
+   **Example response:**
+
+   .. sourcecode:: http
+
+    HTTP/1.1 200 OK
+    Vary: Accept
+    Content-Type: application/json
+
+    {
+      "count": 1,
+      "next": null,
+      "previous": null,
+      "results": [
+        {
+          "attendee": {
+            "fields": [
+              {
+                "id": "attendee_name",
+                "label": "Name",
+                "value": "Jon Doe",
+                "details": [
+                  {"label": "Given name", "value": "John"},
+                  {"label": "Family name", "value": "Doe"},
+                ]
+              },
+              {
+                "id": "attendee_email",
+                "label": "Email",
+                "value": "test@example.com",
+                "details": []
+              }
+           ]
+          },
+          "rating": 4,
+          "tags": ["foo"],
+          "notes": "Great customer, wants our newsletter"
+        }
+      ]
+    }
+
+   :statuscode 200: No error
+   :statuscode 401: Invalid authentication code
+   :statuscode 403: Not permitted to access bulk data

doc/plugins/imported_secrets.rst

@@ -0,0 +1,301 @@
+Secrets Import
+==============
+
+Usually, pretix generates ticket secrets (i.e. the QR code used for scanning) itself. You can read more about this
+process at :ref:`secret_generators`.
+
+With the "Secrets Import" plugin, you can upload your own list of secrets to be used instead. This is useful for
+integrating with third-party check-in systems.
+
+
+API Resource description
+-------------------------
+
+The secrets import plugin provides a HTTP API that allows you to create new secrets.
+
+The imported secret resource contains the following public fields:
+
+.. rst-class:: rest-resource-table
+
+===================================== ========================== =======================================================
+Field                                 Type                       Description
+===================================== ========================== =======================================================
+id                                    integer                    Internal ID of the secret
+secret                                string                     Actual string content of the secret (QR code content)
+used                                  boolean                    Whether the secret was already used for a ticket. If ``true``,
+                                                                 the secret can no longer be deleted. Secrets are never used
+                                                                 twice, even if an order is canceled or deleted.
+item                                  integer                    Internal ID of a product, or ``null``. If set, the secret
+                                                                 will only be used for tickets of this product.
+variation                             integer                    Internal ID of a product variation, or ``null``. If set, the secret
+                                                                 will only be used for tickets of this product variation.
+subevent                              integer                    Internal ID of an event series date, or ``null``. If set, the secret
+                                                                 will only be used for tickets of this event series date.
+===================================== ========================== =======================================================
+
+API Endpoints
+-------------
+
+.. http:get:: /api/v1/organizers/(organizer)/events/(event)/imported_secrets/
+
+   Returns a list of all secrets imported for an event.
+
+   **Example request**:
+
+   .. sourcecode:: http
+
+      GET /api/v1/organizers/bigevents/events/sampleconf/imported_secrets/ HTTP/1.1
+      Host: pretix.eu
+      Accept: application/json, text/javascript
+
+   **Example response**:
+
+   .. sourcecode:: http
+
+      HTTP/1.1 200 OK
+      Vary: Accept
+      Content-Type: application/json
+
+      {
+        "count": 1,
+        "next": null,
+        "previous": null,
+        "results": [
+          {
+            "id": 1,
+            "secret": "foobar",
+            "used": false,
+            "item": null,
+            "variation": null,
+            "subevent": null
+          }
+        ]
+      }
+
+   :query page: The page number in case of a multi-page result set, default is 1
+   :param organizer: The ``slug`` field of a valid organizer
+   :param event: The ``slug`` field of the event to fetch
+   :statuscode 200: no error
+   :statuscode 401: Authentication failure
+   :statuscode 403: The requested organizer or event does not exist **or** you have no permission to view it.
+
+.. http:get:: /api/v1/organizers/(organizer)/events/(event)/imported_secrets/(id)/
+
+   Returns information on one secret, identified by its ID.
+
+   **Example request**:
+
+   .. sourcecode:: http
+
+      GET /api/v1/organizers/bigevents/events/sampleconf/imported_secrets/1/ HTTP/1.1
+      Host: pretix.eu
+      Accept: application/json, text/javascript
+
+   **Example response**:
+
+   .. sourcecode:: http
+
+      HTTP/1.1 200 OK
+      Vary: Accept
+      Content-Type: application/json
+
+      {
+        "id": 1,
+        "secret": "foobar",
+        "used": false,
+        "item": null,
+        "variation": null,
+        "subevent": null
+      }
+
+   :param organizer: The ``slug`` field of the organizer to fetch
+   :param event: The ``slug`` field of the event to fetch
+   :param id: The ``id`` field of the secret to fetch
+   :statuscode 200: no error
+   :statuscode 401: Authentication failure
+   :statuscode 403: The requested organizer/event/secret does not exist **or** you have no permission to view it.
+
+.. http:post:: /api/v1/organizers/(organizer)/events/(event)/imported_secrets/
+
+   Create a new secret.
+
+   **Example request**:
+
+   .. sourcecode:: http
+
+      POST /api/v1/organizers/bigevents/events/sampleconf/imported_secrets/ HTTP/1.1
+      Host: pretix.eu
+      Accept: application/json, text/javascript
+      Content-Type: application/json
+      Content-Length: 166
+
+      {
+        "secret": "foobar",
+        "used": false,
+        "item": null,
+        "variation": null,
+        "subevent": null
+      }
+
+   **Example response**:
+
+   .. sourcecode:: http
+
+      HTTP/1.1 201 Created
+      Vary: Accept
+      Content-Type: application/json
+
+      {
+        "id": 1,
+        "secret": "foobar",
+        "used": false,
+        "item": null,
+        "variation": null,
+        "subevent": null
+      }
+
+   :param organizer: The ``slug`` field of the organizer to a create new secret for
+   :param event: The ``slug`` field of the event to create a new secret for
+   :statuscode 201: no error
+   :statuscode 400: The secret could not be created due to invalid submitted data.
+   :statuscode 401: Authentication failure
+   :statuscode 403: The requested organizer/event does not exist **or** you have no permission to create secrets.
+
+.. http:post:: /api/v1/organizers/(organizer)/events/(event)/imported_secrets/bulk_create/
+
+   Create new secrets in bulk (up to 500 per request). The request either succeeds or fails entirely.
+
+   **Example request**:
+
+   .. sourcecode:: http
+
+      POST /api/v1/organizers/bigevents/events/sampleconf/imported_secrets/bulk_create/ HTTP/1.1
+      Host: pretix.eu
+      Accept: application/json, text/javascript
+      Content-Type: application/json
+      Content-Length: 166
+
+      [
+        {
+          "secret": "foobar",
+          "used": false,
+          "item": null,
+          "variation": null,
+          "subevent": null
+        },
+        {
+          "secret": "baz",
+          "used": false,
+          "item": null,
+          "variation": null,
+          "subevent": null
+        }
+      ]
+
+   **Example response**:
+
+   .. sourcecode:: http
+
+      HTTP/1.1 200 OK
+      Vary: Accept
+      Content-Type: application/json
+
+      [
+        {
+          "id": 1,
+          "secret": "foobar",
+          "used": false,
+          "item": null,
+          "variation": null,
+          "subevent": null
+        },
+        {
+          "id": 2,
+          "secret": "baz",
+          "used": false,
+          "item": null,
+          "variation": null,
+          "subevent": null
+        }
+      ]
+
+   :param organizer: The ``slug`` field of the organizer to create new secrets for
+   :param event: The ``slug`` field of the event to create new secrets for
+   :statuscode 201: no error
+   :statuscode 400: The secrets could not be created due to invalid submitted data.
+   :statuscode 401: Authentication failure
+   :statuscode 403: The requested organizer/event does not exist **or** you have no permission to create secrets.
+
+
+.. http:patch:: /api/v1/organizers/(organizer)/events/(event)/imported_secrets/(id)/
+
+   Update a secret. You can also use ``PUT`` instead of ``PATCH``. With ``PUT``, you have to provide all fields of
+   the resource, other fields will be reset to default. With ``PATCH``, you only need to provide the fields that you
+   want to change.
+
+   **Example request**:
+
+   .. sourcecode:: http
+
+      PATCH /api/v1/organizers/bigevents/events/sampleconf/imported_secrets/1/ HTTP/1.1
+      Host: pretix.eu
+      Accept: application/json, text/javascript
+      Content-Type: application/json
+      Content-Length: 34
+
+      {
+        "item": 2
+      }
+
+   **Example response**:
+
+   .. sourcecode:: http
+
+      HTTP/1.1 200 OK
+      Vary: Accept
+      Content-Type: text/javascript
+
+      {
+        "id": 1,
+        "secret": "foobar",
+        "used": false,
+        "item": 2,
+        "variation": null,
+        "subevent": null
+      }
+
+   :param organizer: The ``slug`` field of the organizer to modify
+   :param event: The ``slug`` field of the event to modify
+   :param id: The ``id`` field of the secret to modify
+   :statuscode 200: no error
+   :statuscode 400: The secret could not be modified due to invalid submitted data.
+   :statuscode 401: Authentication failure
+   :statuscode 403: The requested organizer/event/secret does not exist **or** you have no permission to change it.
+
+
+.. http:delete:: /api/v1/organizers/(organizer)/events/(event)/imported_secrets/(id)/
+
+   Delete a secret. You can only delete secrets that have not yet been used.
+
+   **Example request**:
+
+   .. sourcecode:: http
+
+      DELETE /api/v1/organizers/bigevents/events/sampleconf/imported_secrets/1/ HTTP/1.1
+      Host: pretix.eu
+      Accept: application/json, text/javascript
+
+   **Example response**:
+
+   .. sourcecode:: http
+
+      HTTP/1.1 204 No Content
+      Vary: Accept
+
+   :param organizer: The ``slug`` field of the organizer to modify
+   :param event: The ``slug`` field of the event to modify
+   :param id: The ``id`` field of the secret to delete
+   :statuscode 204: no error
+   :statuscode 401: Authentication failure
+   :statuscode 403: The requested organizer/event/secret does not exist **or** you have no permission to change it **or** the secret has already been used
+

doc/plugins/index.rst

@@ -1,3 +1,5 @@
+.. _`plugin-docs`:
+
 Plugin documentation
 ====================
 
@@ -10,11 +12,13 @@ If you want to **create** a plugin, please go to the
    :maxdepth: 2
 
    list
-   pretixdroid
    banktransfer
    ticketoutputpdf
    badges
    campaigns
    certificates
    digital
+   exhibitors
+   imported_secrets
    webinar
+   presale-saml

doc/plugins/list.rst

@@ -5,6 +5,6 @@ List of plugins
 ===============
 
 A detailed list of plugins that are available for pretix can be found on the
-`project website`_.
+`pretix Marketplace`_.
 
-.. _project website: https://pretix.eu/about/en/plugins
+.. _pretix Marketplace: https://marketplace.pretix.eu

doc/plugins/presale-saml.rst

@@ -0,0 +1,405 @@
+.. highlight:: ini
+.. spelling::
+
+   IdP
+   skIDentity
+   ePA
+   NPA
+
+Presale SAML Authentication
+===========================
+
+The Presale SAML Authentication plugin is an advanced plugin, which most event
+organizers will not need to use. However, for the select few  who do require
+strong customer authentication that cannot be covered by the built-in customer
+account functionality, this plugin allows pretix to connect to a SAML IdP and
+perform authentication and retrieval of user information.
+
+Usage of the plugin is governed by two separate sets of settings: The plugin
+installation, the Service Provider (SP) configuration and the event
+configuration.
+
+Plugin installation and initial configuration
+---------------------------------------------
+
+.. note:: If you are a customer of our hosted `pretix.eu`_ offering, you can
+          skip this section.
+
+The plugin is installed as any other plugin in the pretix ecosystem. As a
+pretix system administrator, please follow the instructions in the the
+:ref:`Administrator documentation <admindocs>`.
+
+Once installed, you will need to assess, if you want (or need) your pretix
+instance to be a single SP for all organizers and events or if every event
+organizer has to provide their own SP.
+
+Take the example of a university which runs pretix under an pretix Enterprise
+agreement. Since they only provide ticketing services to themselves (every
+organizer is still just a different department of the same university), a
+single SP should be enough.
+
+On the other hand, a reseller such as `pretix.eu`_ who services a multitude
+of clients would not work that way. Here, every organizer is a separate
+legal entity and as such will also need to provide their own SP configuration:
+Company A will expect their SP to reflect their company - and not a generalized
+"pretix SP".
+
+Once you have decided on the mode of operation, the :ref:`Configuration file
+<config>` needs to be extended to reflect your choice.
+
+Example::
+
+    [presale-saml]
+    level=global
+
+``level``
+    ``global`` to use only a single, system-wide SP, ``organizer`` for multiple
+    SPs, configured on the organizer-level. Defaults to ``organizer``.
+
+Service Provider configuration
+------------------------------
+
+Global Level
+^^^^^^^^^^^^
+
+.. note:: If you are a customer of our hosted `pretix.eu`_ offering, you can
+          skip this section and follow the instructions on the upcoming
+          Organizer Level settings.
+
+As a user with administrative privileges, please activate them by clicking the
+`Admin Mode` button in the top right hand corner.
+
+You should now see a new menu-item titled `SAML` appear.
+
+Organizer Level
+^^^^^^^^^^^^^^^
+
+Navigate to the organizer settings in the pretix backend. In the navigation
+bar, you will find a menu-item titled `SAML` if your user has the `Can
+change organizer settings` permission.
+
+
+.. note:: If you are a customer of our hosted `pretix.eu`_ offering, the menu
+          will only appear once one of our friendly customer service agents
+          has enabled the Presale SAML Authentication plugin for at least one
+          of your events. Feel free to get in touch with us!
+
+Setting up the SP
+^^^^^^^^^^^^^^^^^
+
+No matter where your SP configuration lives, you will be greeted by a very
+long list of fields of which almost all of them will need to be filled. Please
+don't be discouraged - most of the settings don't need to be decided by yourself
+and/or are already preset with a sensible default setting.
+
+If you are not sure what setting you should choose for any of the fields, you
+should reach out to your IdP operator as they can tell you exactly what the IdP
+expects and - more importantly - supports.
+
+``IdP Metadata URL``
+    Please provide the URL where your IdP outputs its metadata. For most IdPs,
+    this URL is static and the same for all SPs. If you are a member of the
+    DFN-AAI, you can find the meta-data for the `Test-, Basic- and
+    Advanced-Federation`_ on their website. Please do talk with your local
+    IdP operator though, as you might not even need to go through the DFN-AAI
+    and might just use your institutions local IdP which will also host their
+    metadata on a different URL.
+
+    The URL needs to be publicly accessible, as saving the settings form will
+    fail if the IdP metadata cannot be retrieved. pretix will also automatically
+    refresh the IdP metadata on a regular basis.
+
+``SP Entity Id``
+    By default, we recommend that you use the system-proposed metadata-URL as
+    the Entity Id of your SP. However, if so desired or required by your IdP,
+    you can also set any other, arbitrary URL as the SP Entity Id.
+
+``SP Name / SP Decription``
+    Most IdP will display the name and description of your SP to the users
+    during authentication. The description field can be used to explain to the
+    users how their data is being used.
+
+``SP X.509 Certificate / SP X.509 Private Key``
+    Your SP needs a certificate and a private key for said certificate. Please
+    coordinate with your IdP, if you are supposed to generate these yourself or
+    if they are provided to you.
+
+``SP X.509 New Certificate``
+    As certificates have an expiry date, they need to be renewed on a regular
+    basis. In order to facilitate the rollover from the expiring to the new
+    certificate, you can provide the new certificate already before the expiration
+    of the existing one. That way, the system will automatically use the correct
+    one. Once the old certificate has expired and is not used anymore at all,
+    you can move the new certificate into the slot of the normal certificate and
+    keep the new slot empty for your next renewal process.
+
+``Requested Attributes``
+    An IdP can hold a variety of attributes of an authenticating user. While
+    your IdP will dictate which of the available attributes your SP can consume
+    in theory, you will still need to define exactly which attributes the SP
+    should request.
+
+    The notation is a JSON list of objects with 5 attributes each:
+
+      * ``attributeValue``: Can be defaulted to ``[]``.
+      * ``friendlyName``: String used in the upcoming event-level settings to
+        retrieve the attributes data.
+      * ``isRequired``: Boolean indicating whether the IdP must enforce the
+        transmission of this attribute. In most cases, ``true`` is the best
+        choice.
+      * ``name``: String of the internal, technical name of the requested
+        attribute. Often starting with ``urn:mace:dir:attribute-def:``,
+        ``urn:oid:`` or ``http://``/``https://``.
+      * ``nameFormat``: String describing the type of ``name`` that has been
+        set in the previous section. Often starting with
+        ``urn:mace:shibboleth:1.0:`` or ``urn:oasis:names:tc:SAML:2.0:``.
+
+    Your IdP can provide you with a list of available attributes. See below
+    for a sample configuration in an academic context.
+
+    Note, that you can have multiple attributes with the same ``friendlyName``
+    but different ``name``s. This is often used in systems, where the same
+    information (for example a persons name) is saved in different fields -
+    for example because one institution is returning SAML 1.0 and other
+    institutions are returning SAML 2.0 style attributes. Typically, this only
+    occurs in mix environments like the DFN-AAI with a large number of
+    participants. If you are only using your own institutions IdP and not
+    authenticating anyone outside of your realm, this should not be a common
+    sight.
+
+``Encrypt/Sign/Require ...``
+    Does what is says on the box - please inquire with your IdP for the
+    necessary settings. Most settings can be turned on as they increase security,
+    however some IdPs might stumble over some of them.
+
+``Signature / Digest Algorithm``
+    Please chose appropriate algorithms, that both pretix/your SP and the IdP
+    can communicate with. A common source of issues when connecting to a
+    Shibboleth-based IdP is the Digest Algorithm: pretix does not support
+    ``http://www.w3.org/2009/xmlenc11#rsa-oaep`` and authentication will fail
+    if the IdP enforces this.
+
+``Technical/Support Contacts``
+    Those contacts are encoded into the SPs public meta data and might be
+    displayed to users having trouble authenticating. It is recommended to
+    provide a dedicated point of contact for technical issues, as those will
+    be the ones to change the configuration for the SP.
+
+Event / Authentication configuration
+------------------------------------
+
+Basic settings
+^^^^^^^^^^^^^^
+
+Once the plugin has been enabled for a pretix event using the Plugins-menu from
+the event's settings, a new *SAML* menu item will show up.
+
+On this page, the actual authentication can be configured.
+
+``Checkout Explanation``
+    Since most users probably won't be familiar with why they have to authenticate
+    to buy a ticket, you can provide them a small blurb here. Markdown is supported.
+
+``Attribute RegEx``
+    By default, any successful authentication with the IdP will allow the user to
+    proceed with their purchase. Should the allowed audience needed to be restricted
+    further, a set of regular Expressions can be used to do this.
+
+    An Attribute RegEx of ``{}`` will allow any authenticated user to pass.
+
+    A RegEx of ``{ "affiliation": "^(employee@pretix.eu|staff@pretix.eu)$" }`` will
+    only allow user to pass which have the ``affiliation`` attribute and whose
+    attribute either matches ``employee@pretix.eu`` or ``staff@pretix.eu``.
+
+    Please make sure that the attribute you are querying is also requested from the
+    IdP in the first place - for a quick check you can have a look at the top of
+    the page where all currently configured attributes are listed.
+
+``RegEx Fail Explanation``
+    Only used in conjunction with the above Attribute RegEx. Should the user not
+    pass the restrictions imposed by the regular expression, the user is shown
+    this error-message.
+
+    If you are - for example in an university context - restricting access to
+    students only, you might want to explain here that Employees are not allowed
+    to book tickets.
+
+``Ticket Secret SAML Attribute``
+    In very specific instances, it might be desirable that the ticket-secret is
+    not the randomly one generated by pretix but rather based on one of the
+    users attributes - for example their unique ID or access card number.
+
+    To achieve this, the name of a SAML-attribute can be specified here.
+
+    It is however necessary to note, that even with this setting in use,
+    ticket-secrets need to be unique. This is why when this setting is enabled,
+    the default, pretix-generated ticket-secret is prefixed with the attributes
+    value.
+
+    Example: A users ``cardid`` attribute has the value of ``01189998819991197253``.
+    The default random ticket secret would have been
+    ``yczygpw9877akz2xwdhtdyvdqwkv7npj``. The resulting new secret will now be
+    ``01189998819991197253_yczygpw9877akz2xwdhtdyvdqwkv7npj``.
+
+    That way, the ticket secret is still unique, but when checking into an event,
+    the user can easily be searched and found using their identifier.
+
+IdP-provided E-Mail addresses, names
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+By default, pretix will only authenticate the user and not process the received
+data any further.
+
+However, there are a few exceptions to this rule.
+
+There are a few `magic` attributes that pretix will use to automatically populate
+the corresponding fields within the checkout process **and lock them out from
+user editing**.
+
+  * ``givenName`` and ``sn``: If both of those attributes are present and pretix
+    is configured to collect the users name, these attributes' values are used
+    for the given and family name respectively.
+  * ``email``: If this attribute is present, the E-Mail-address of the users will
+    be set to the one transmitted through the attributes.
+
+The latter might pose a problem, if the IdP is transmitting an ``email`` attribute
+which does contain a system-level mail address which is only used as an internal
+identifier but not as a real mailbox. In this case, please consider setting the
+``friendlyName`` of the attribute to a different value than ``email`` or removing
+this field from the list of requested attributes altogether.
+
+Saving attributes to questions
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+By setting the ``internal identifier`` of a user-defined question to the same name
+as a SAML attribute, pretix will save the value of said attribute into the question.
+
+All the same as in the above section on E-Mail addresses, those fields become
+non-editable by the user.
+
+Please be aware that some specialty question types might not be compatible with
+the SAML attributes due to specific format requirements. If in doubt (or if the
+checkout fails/the information is not properly saved), try setting the question
+type to a simple type like "Text (one line)".
+
+Notes and configuration examples
+--------------------------------
+
+Requesting SAML 1.0 and 2.0 attributes from an academic IdP
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+This requests the ``eduPersonPrincipalName`` (also sometimes called EPPN),
+``email``, ``givenName`` and ``sn`` both in SAML 1.0 and SAML 2.0 attributes.
+
+.. sourcecode:: json
+
+    [
+        {
+            "attributeValue": [],
+            "friendlyName": "eduPersonPrincipalName",
+            "isRequired": true,
+            "name": "urn:mace:dir:attribute-def:eduPersonPrincipalName",
+            "nameFormat": "urn:mace:shibboleth:1.0:attributeNamespace:uri"
+        },
+        {
+            "attributeValue": [],
+            "friendlyName": "eduPersonPrincipalName",
+            "isRequired": true,
+            "name": "urn:oid:1.3.6.1.4.1.5923.1.1.1.6",
+            "nameFormat": "urn:oasis:names:tc:SAML:2.0:attrname-format:uri"
+        },
+        {
+            "attributeValue": [],
+            "friendlyName": "email",
+            "isRequired": true,
+            "name": "urn:mace:dir:attribute-def:mail",
+            "nameFormat": "urn:mace:shibboleth:1.0:attributeNamespace:uri"
+        },
+        {
+            "attributeValue": [],
+            "friendlyName": "email",
+            "isRequired": true,
+            "name": "urn:oid:0.9.2342.19200300.100.1.3",
+            "nameFormat": "urn:oasis:names:tc:SAML:2.0:attrname-format:uri"
+        },
+        {
+            "attributeValue": [],
+            "friendlyName": "givenName",
+            "isRequired": true,
+            "name": "urn:mace:dir:attribute-def:givenName",
+            "nameFormat": "urn:mace:shibboleth:1.0:attributeNamespace:uri"
+        },
+        {
+            "attributeValue": [],
+            "friendlyName": "givenName",
+            "isRequired": true,
+            "name": "urn:oid:2.5.4.42",
+            "nameFormat": "urn:oasis:names:tc:SAML:2.0:attrname-format:uri"
+        },
+        {
+            "attributeValue": [],
+            "friendlyName": "sn",
+            "isRequired": true,
+            "name": "urn:mace:dir:attribute-def:sn",
+            "nameFormat": "urn:mace:shibboleth:1.0:attributeNamespace:uri"
+        },
+        {
+            "attributeValue": [],
+            "friendlyName": "sn",
+            "isRequired": true,
+            "name": "urn:oid:2.5.4.4",
+            "nameFormat": "urn:oasis:names:tc:SAML:2.0:attrname-format:uri"
+        }
+    ]
+
+skIDentity IdP Metadata URL
+^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Since the IdP Metadata URL for `skIDentity`_ is not readily documented/visible
+in their backend, we document it here:
+``https://service.skidentity.de/fs/saml/metadata``
+
+Requesting skIDentity attributes for electronic identity cards
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+This requests the basic ``eIdentifier``, ``IDType``, ``IDIssuer``, and
+``NameID`` from the `skIDentity`_ SAML service, which are available for
+electronic ID cards such as the German ePA/NPA. (Other attributes such as
+the name and address are available at additional cost from the IdP).
+
+.. sourcecode:: json
+
+    [
+        {
+            "attributeValue": [],
+            "friendlyName": "eIdentifier",
+            "isRequired": true,
+            "name": "http://www.skidentity.de/att/eIdentifier",
+            "nameFormat": "urn:oasis:names:tc:SAML:2.0:attrname-format:uri"
+        },
+        {
+            "attributeValue": [],
+            "friendlyName": "IDType",
+            "isRequired": true,
+            "name": "http://www.skidentity.de/att/IDType",
+            "nameFormat": "urn:oasis:names:tc:SAML:2.0:attrname-format:uri"
+        },
+        {
+            "attributeValue": [],
+            "friendlyName": "IDIssuer",
+            "isRequired": true,
+            "name": "http://www.skidentity.de/att/IDIssuer",
+            "nameFormat": "urn:oasis:names:tc:SAML:2.0:attrname-format:uri"
+        },
+        {
+            "attributeValue": [],
+            "friendlyName": "NameID",
+            "isRequired": true,
+            "name": "http://www.skidentity.de/att/NameID",
+            "nameFormat": "urn:oasis:names:tc:SAML:2.0:attrname-format:uri"
+        }
+    ]
+
+.. _pretix.eu: https://pretix.eu
+.. _Test-, Basic- and Advanced-Federation: https://doku.tid.dfn.de/en:metadata
+.. _skIDentity: https://www.skidentity.de/

doc/plugins/pretixdroid.rst

@@ -1,368 +0,0 @@
-pretixdroid HTTP API
-====================
-
-The pretixdroid plugin provides a HTTP API that the `pretixdroid Android app`_
-uses to communicate with the pretix server.
-
-.. warning:: This API is **DEPRECATED** and will probably go away soon. It is used **only** to serve the pretixdroid
-             Android app. There are no backwards compatibility guarantees on this API. We will not add features that
-             are not required for the  Android App. There is a general-purpose :ref:`rest-api` that provides all
-             features that you need to check in.
-
-.. versionchanged:: 1.12
-
-   Support for check-in-time questions has been added. The new API features are fully backwards-compatible and
-   negotiated live, so clients which do not need this feature can ignore the change. For this reason, the API version
-   has not been increased and is still set to 3.
-
-.. versionchanged:: 1.13
-
-   Support for checking in unpaid tickets has been added.
-
-
-.. http:post:: /pretixdroid/api/(organizer)/(event)/redeem/
-
-   Redeems a ticket, i.e. checks the user in.
-
-   **Example request**:
-
-   .. sourcecode:: http
-
-      POST /pretixdroid/api/demoorga/democon/redeem/?key=ABCDEF HTTP/1.1
-      Host: demo.pretix.eu
-      Accept: application/json, text/javascript
-      Content-Type: application/x-www-form-urlencoded
-
-      secret=az9u4mymhqktrbupmwkvv6xmgds5dk3&questions_supported=true
-
-   You **must** set the parameter secret.
-
-   You **must** set the parameter ``questions_supported`` to ``true`` **if** you support asking questions
-   back to the app operator. You **must not** set it if you do not support this feature. In that case, questions
-   will just be ignored.
-
-   You **may** set the additional parameter ``datetime`` in the body containing an ISO8601-encoded
-   datetime of the entry attempt. If you don"t, the current date and time will be used.
-
-   You **may** set the additional parameter ``force`` to indicate that the request should be logged
-   regardless of previous check-ins for the same ticket. This might be useful if you made the entry decision offline.
-   Questions will also always be ignored in this case (i.e. supplied answers will be saved, but no error will be
-   thrown if they are missing or invalid).
-
-   You **may** set the additional parameter ``nonce`` with a globally unique random value to identify this
-   check-in. This is meant to be used to prevent duplicate check-ins when you are just retrying after a connection
-   failure.
-
-   You **may** set the additional parameter ``ignore_unpaid`` to indicate that the check-in should be performed even
-   if the order is in pending state.
-
-   If questions are supported and required, you will receive a dictionary ``questions`` containing details on the
-   particular questions to ask. To answer them, just re-send your redemption request with additional parameters of
-   the form ``answer_<question>=<answer>``, e.g. ``answer_12=24``.
-
-   **Example successful response**:
-
-   .. sourcecode:: http
-
-      HTTP/1.1 200 OK
-      Content-Type: text/json
-
-      {
-        "status": "ok"
-        "version": 3,
-        "data": {
-          "secret": "az9u4mymhqktrbupmwkvv6xmgds5dk3",
-          "order": "ABCDE",
-          "item": "Standard ticket",
-          "item_id": 1,
-          "variation": null,
-          "variation_id": null,
-          "attendee_name": "Peter Higgs",
-          "attention": false,
-          "redeemed": true,
-          "checkin_allowed": true,
-          "addons_text": "Parking spot",
-          "paid": true
-        }
-      }
-
-   **Example response with required questions**:
-
-   .. sourcecode:: http
-
-      HTTP/1.1 200 OK
-      Content-Type: text/json
-
-      {
-        "status": "incomplete"
-        "version": 3
-        "data": {
-          "secret": "az9u4mymhqktrbupmwkvv6xmgds5dk3",
-          "order": "ABCDE",
-          "item": "Standard ticket",
-          "item_id": 1,
-          "variation": null,
-          "variation_id": null,
-          "attendee_name": "Peter Higgs",
-          "attention": false,
-          "redeemed": true,
-          "checkin_allowed": true,
-          "addons_text": "Parking spot",
-          "paid": true
-        },
-        "questions": [
-          {
-            "id": 12,
-            "type": "C",
-            "question": "Choose a shirt size",
-            "required": true,
-            "position": 2,
-            "items": [1],
-            "options": [
-              {
-                "id": 24,
-                "answer": "M"
-              },
-              {
-                "id": 25,
-                "answer": "L"
-              }
-            ]
-          }
-        ]
-      }
-
-   **Example error response with data**:
-
-   .. sourcecode:: http
-
-      HTTP/1.1 200 OK
-      Content-Type: text/json
-
-      {
-        "status": "error",
-        "reason": "already_redeemed",
-        "version": 3,
-        "data": {
-          "secret": "az9u4mymhqktrbupmwkvv6xmgds5dk3",
-          "order": "ABCDE",
-          "item": "Standard ticket",
-          "item_id": 1,
-          "variation": null,
-          "variation_id": null,
-          "attendee_name": "Peter Higgs",
-          "attention": false,
-          "redeemed": true,
-          "checkin_allowed": true,
-          "addons_text": "Parking spot",
-          "paid": true
-        }
-      }
-
-   **Example error response without data**:
-
-   .. sourcecode:: http
-
-      HTTP/1.1 200 OK
-      Content-Type: text/json
-
-      {
-        "status": "error",
-        "reason": "unkown_ticket",
-        "version": 3
-      }
-
-   Possible error reasons:
-
-   * ``unpaid`` - Ticket is not paid for or has been refunded
-   * ``already_redeemed`` - Ticket already has been redeemed
-   * ``product`` - Tickets with this product may not be scanned at this device
-   * ``unknown_ticket`` - Secret does not match a ticket in the database
-
-   :query key: Secret API key
-   :statuscode 200: Valid request
-   :statuscode 404: Unknown organizer or event
-   :statuscode 403: Invalid authorization key
-
-.. http:get:: /pretixdroid/api/(organizer)/(event)/search/
-
-   Searches for a ticket.
-   At most 25 results will be returned. **Queries with less than 4 characters will always return an empty result set.**
-
-   **Example request**:
-
-   .. sourcecode:: http
-
-      GET /pretixdroid/api/demoorga/democon/search/?key=ABCDEF&query=Peter HTTP/1.1
-      Host: demo.pretix.eu
-      Accept: application/json, text/javascript
-
-   **Example response**:
-
-   .. sourcecode:: http
-
-      HTTP/1.1 200 OK
-      Content-Type: text/json
-
-      {
-        "results": [
-          {
-            "secret": "az9u4mymhqktrbupmwkvv6xmgds5dk3",
-            "order": "ABCE6",
-            "item": "Standard ticket",
-            "variation": null,
-            "attendee_name": "Peter Higgs",
-            "redeemed": false,
-            "attention": false,
-            "checkin_allowed": true,
-            "addons_text": "Parking spot",
-            "paid": true
-          },
-          ...
-        ],
-        "version": 3
-      }
-
-   :query query: Search query
-   :query key: Secret API key
-   :statuscode 200: Valid request
-   :statuscode 404: Unknown organizer or event
-   :statuscode 403: Invalid authorization key
-
-.. http:get:: /pretixdroid/api/(organizer)/(event)/download/
-
-   Download data for all tickets.
-
-   **Example request**:
-
-   .. sourcecode:: http
-
-      GET /pretixdroid/api/demoorga/democon/download/?key=ABCDEF HTTP/1.1
-      Host: demo.pretix.eu
-      Accept: application/json, text/javascript
-
-   **Example response**:
-
-   .. sourcecode:: http
-
-      HTTP/1.1 200 OK
-      Content-Type: text/json
-
-      {
-        "version": 3,
-        "results": [
-          {
-            "secret": "az9u4mymhqktrbupmwkvv6xmgds5dk3",
-            "order": "ABCE6",
-            "item": "Standard ticket",
-            "variation": null,
-            "attendee_name": "Peter Higgs",
-            "redeemed": false,
-            "attention": false,
-            "checkin_allowed": true,
-            "paid": true
-          },
-          ...
-        ],
-        "questions": [
-          {
-            "id": 12,
-            "type": "C",
-            "question": "Choose a shirt size",
-            "required": true,
-            "position": 2,
-            "items": [1],
-            "options": [
-              {
-                "id": 24,
-                "answer": "M"
-              },
-              {
-                "id": 25,
-                "answer": "L"
-              }
-            ]
-          }
-        ]
-      }
-
-   :query key: Secret API key
-   :statuscode 200: Valid request
-   :statuscode 404: Unknown organizer or event
-   :statuscode 403: Invalid authorization key
-
-.. http:get:: /pretixdroid/api/(organizer)/(event)/status/
-
-   Returns status information, such as the total number of tickets and the
-   number of performed check-ins.
-
-   **Example request**:
-
-   .. sourcecode:: http
-
-      GET /pretixdroid/api/demoorga/democon/status/?key=ABCDEF HTTP/1.1
-      Host: demo.pretix.eu
-      Accept: application/json, text/javascript
-
-   **Example response**:
-
-   .. sourcecode:: http
-
-      HTTP/1.1 200 OK
-      Content-Type: text/json
-
-      {
-        "checkins": 17,
-        "total": 42,
-        "version": 3,
-        "event": {
-          "name": "Demo Conference",
-          "slug": "democon",
-          "date_from": "2016-12-27T17:00:00Z",
-          "date_to": "2016-12-30T18:00:00Z",
-          "timezone": "UTC",
-          "url": "https://demo.pretix.eu/demoorga/democon/",
-          "organizer": {
-            "name": "Demo Organizer",
-            "slug": "demoorga"
-          },
-        },
-        "items": [
-          {
-            "name": "T-Shirt",
-            "id": 1,
-            "checkins": 1,
-            "admission": False,
-            "total": 1,
-            "variations": [
-              {
-                "name": "Red",
-                "id": 1,
-                "checkins": 1,
-                "total": 12
-              },
-              {
-               "name": "Blue",
-                "id": 2,
-                "checkins": 4,
-                "total": 8
-              }
-            ]
-          },
-          {
-            "name": "Ticket",
-            "id": 2,
-            "checkins": 15,
-            "admission": True,
-            "total": 22,
-            "variations": []
-          }
-        ]
-      }
-
-   :query key: Secret API key
-   :statuscode 200: Valid request
-   :statuscode 404: Unknown organizer or event
-   :statuscode 403: Invalid authorization key
-
-.. _pretixdroid Android app: https://github.com/pretix/pretixdroid

doc/requirements.txt

@@ -1,5 +1,6 @@
 -e ../src/
 sphinx==2.3.*
+jinja2==3.0.*
 sphinx-rtd-theme
 sphinxcontrib-httpdomain
 sphinxcontrib-images

doc/spelling_wordlist.txt

@@ -103,6 +103,7 @@ prepending
 preprocessor
 presale
 pretix
+pretixLEAD
 pretixSCAN
 pretixdroid
 pretixPOS

doc/user/events/ticket_secrets.rst

@@ -1,3 +1,5 @@
+.. _secret_generators:
+
 Ticket secret generators
 ========================
 

doc/user/events/widget.rst

@@ -253,18 +253,21 @@ If you want, you can suppress us loading the widget and/or modify the user data
 
 If you then later want to trigger loading the widgets, just call ``window.PretixWidget.buildWidgets()``.
 
-Waiting for the widget to load
-------------------------------
+Waiting for the widget to load or close
+---------------------------------------
 
-If you want to run custom JavaScript once the widget is fully loaded, you can register a callback function. Note that
-this function might be run multiple times, for example if you have multiple widgets on a page or if the user switches
-e.g. from an event list to an event detail view::
+If you want to run custom JavaScript once the widget is fully loaded or when it is closed, you can register callback
+functions. Note that these function might be run multiple times, for example if you have multiple widgets on a page
+or if the user switches e.g. from an event list to an event detail view::
 
     <script type="text/javascript">
     window.pretixWidgetCallback = function () {
         window.PretixWidget.addLoadListener(function () {
             console.log("Widget has loaded!");
         });
+        window.PretixWidget.addCloseListener(function () {
+            console.log("Widget has been closed!");
+        });
     }
     </script>
 

src/MANIFEST.in

@@ -13,6 +13,7 @@ recursive-include pretix/plugins/banktransfer/static *
 recursive-include pretix/plugins/manualpayment/templates *
 recursive-include pretix/plugins/manualpayment/static *
 recursive-include pretix/plugins/paypal/templates *
+recursive-include pretix/plugins/paypal/static *
 recursive-include pretix/plugins/pretixdroid/templates *
 recursive-include pretix/plugins/pretixdroid/static *
 recursive-include pretix/plugins/sendmail/templates *

src/pretix/__init__.py

@@ -19,4 +19,4 @@
 # You should have received a copy of the GNU Affero General Public License along with this program.  If not, see
 # <https://www.gnu.org/licenses/>.
 #
-__version__ = "4.6.0"
+__version__ = "4.9.0.dev0"

src/pretix/api/auth/devicesecurity.py

@@ -167,6 +167,8 @@ class PretixPosSecurityProfile(AllowListSecurityProfile):
         ('GET', 'api-v1:checkinlist-list'),
         ('POST', 'api-v1:checkinlistpos-redeem'),
         ('POST', 'plugins:pretix_posbackend:order.posprintlog'),
+        ('POST', 'plugins:pretix_posbackend:order.poslock'),
+        ('DELETE', 'plugins:pretix_posbackend:order.poslock'),
         ('DELETE', 'api-v1:cartposition-detail'),
         ('GET', 'api-v1:giftcard-list'),
         ('POST', 'api-v1:giftcard-transact'),
@@ -174,8 +176,11 @@ class PretixPosSecurityProfile(AllowListSecurityProfile):
         ('POST', 'plugins:pretix_posbackend:posreceipt-list'),
         ('POST', 'plugins:pretix_posbackend:posclosing-list'),
         ('POST', 'plugins:pretix_posbackend:posdebugdump-list'),
+        ('POST', 'plugins:pretix_posbackend:posdebuglogentry-list'),
+        ('POST', 'plugins:pretix_posbackend:posdebuglogentry-bulk-create'),
         ('GET', 'plugins:pretix_posbackend:poscashier-list'),
         ('POST', 'plugins:pretix_posbackend:stripeterminal.token'),
+        ('PUT', 'plugins:pretix_posbackend:file.upload'),
         ('GET', 'api-v1:revokedsecrets-list'),
         ('GET', 'api-v1:event.settings'),
         ('GET', 'plugins:pretix_seating:event.event'),

src/pretix/api/serializers/checkin.py

@@ -60,7 +60,7 @@ class CheckinListSerializer(I18nAwareModelSerializer):
         full_data = self.to_internal_value(self.to_representation(self.instance)) if self.instance else {}
         full_data.update(data)
 
-        for item in full_data.get('limit_products'):
+        for item in full_data.get('limit_products', []):
             if event != item.event:
                 raise ValidationError(_('One or more items do not belong to this event.'))
 

src/pretix/api/serializers/item.py

@@ -251,9 +251,12 @@ class ItemSerializer(I18nAwareModelSerializer):
         bundles_data = validated_data.pop('bundles') if 'bundles' in validated_data else {}
         meta_data = validated_data.pop('meta_data', None)
         picture = validated_data.pop('picture', None)
+        require_membership_types = validated_data.pop('require_membership_types', [])
         item = Item.objects.create(**validated_data)
         if picture:
             item.picture.save(os.path.basename(picture.name), picture)
+        if require_membership_types:
+            item.require_membership_types.add(*require_membership_types)
 
         for variation_data in variations_data:
             require_membership_types = variation_data.pop('require_membership_types', [])

src/pretix/api/serializers/order.py

@@ -424,88 +424,7 @@ class OrderPositionSerializer(I18nAwareModelSerializer):
             self.fields.pop('pdf_data', None)
 
     def validate(self, data):
-        if data.get('attendee_name') and data.get('attendee_name_parts'):
-            raise ValidationError(
-                {'attendee_name': ['Do not specify attendee_name if you specified attendee_name_parts.']}
-            )
-        if data.get('attendee_name_parts') and '_scheme' not in data.get('attendee_name_parts'):
-            data['attendee_name_parts']['_scheme'] = self.context['request'].event.settings.name_scheme
-
-        if data.get('country'):
-            if not pycountry.countries.get(alpha_2=data.get('country').code):
-                raise ValidationError(
-                    {'country': ['Invalid country code.']}
-                )
-
-        if data.get('state'):
-            cc = str(data.get('country') or self.instance.country or '')
-            if cc not in COUNTRIES_WITH_STATE_IN_ADDRESS:
-                raise ValidationError(
-                    {'state': ['States are not supported in country "{}".'.format(cc)]}
-                )
-            if not pycountry.subdivisions.get(code=cc + '-' + data.get('state')):
-                raise ValidationError(
-                    {'state': ['"{}" is not a known subdivision of the country "{}".'.format(data.get('state'), cc)]}
-                )
-        return data
-
-    def update(self, instance, validated_data):
-        # Even though all fields that shouldn't be edited are marked as read_only in the serializer
-        # (hopefully), we'll be extra careful here and be explicit about the model fields we update.
-        update_fields = [
-            'attendee_name_parts', 'company', 'street', 'zipcode', 'city', 'country',
-            'state', 'attendee_email',
-        ]
-        answers_data = validated_data.pop('answers', None)
-
-        name = validated_data.pop('attendee_name', '')
-        if name and not validated_data.get('attendee_name_parts'):
-            validated_data['attendee_name_parts'] = {
-                '_legacy': name
-            }
-
-        for attr, value in validated_data.items():
-            if attr in update_fields:
-                setattr(instance, attr, value)
-
-        instance.save(update_fields=update_fields)
-
-        if answers_data is not None:
-            qs_seen = set()
-            answercache = {
-                a.question_id: a for a in instance.answers.all()
-            }
-            for answ_data in answers_data:
-                options = answ_data.pop('options', [])
-                if answ_data['question'].pk in qs_seen:
-                    raise ValidationError(f'Question {answ_data["question"]} was sent twice.')
-                if answ_data['question'].pk in answercache:
-                    a = answercache[answ_data['question'].pk]
-                    if isinstance(answ_data['answer'], File):
-                        a.file.save(answ_data['answer'].name, answ_data['answer'], save=False)
-                        a.answer = 'file://' + a.file.name
-                    elif a.answer.startswith('file://') and answ_data['answer'] == "file:keep":
-                        pass  # keep current file
-                    else:
-                        for attr, value in answ_data.items():
-                            setattr(a, attr, value)
-                    a.save()
-                else:
-                    if isinstance(answ_data['answer'], File):
-                        an = answ_data.pop('answer')
-                        a = instance.answers.create(**answ_data, answer='')
-                        a.file.save(os.path.basename(an.name), an, save=False)
-                        a.answer = 'file://' + a.file.name
-                        a.save()
-                    else:
-                        a = instance.answers.create(**answ_data)
-                a.options.set(options)
-                qs_seen.add(a.question_id)
-            for qid, a in answercache.items():
-                if qid not in qs_seen:
-                    a.delete()
-
-        return instance
+        raise TypeError("this serializer is readonly")
 
 
 class RequireAttentionField(serializers.Field):
@@ -593,7 +512,7 @@ class OrderPaymentDateField(serializers.DateField):
 class OrderFeeSerializer(I18nAwareModelSerializer):
     class Meta:
         model = OrderFee
-        fields = ('fee_type', 'value', 'description', 'internal_type', 'tax_rate', 'tax_value', 'tax_rule', 'canceled')
+        fields = ('id', 'fee_type', 'value', 'description', 'internal_type', 'tax_rate', 'tax_value', 'tax_rule', 'canceled')
 
 
 class PaymentURLField(serializers.URLField):
@@ -934,7 +853,8 @@ class OrderCreateSerializer(I18nAwareModelSerializer):
     consume_carts = serializers.ListField(child=serializers.CharField(), required=False)
     force = serializers.BooleanField(default=False, required=False)
     payment_date = serializers.DateTimeField(required=False, allow_null=True)
-    send_email = serializers.BooleanField(default=False, required=False)
+    send_email = serializers.BooleanField(default=False, required=False, allow_null=True)
+    require_approval = serializers.BooleanField(default=False, required=False)
     simulate = serializers.BooleanField(default=False, required=False)
     customer = serializers.SlugRelatedField(slug_field='identifier', queryset=Customer.objects.none(), required=False)
 
@@ -947,7 +867,7 @@ class OrderCreateSerializer(I18nAwareModelSerializer):
         model = Order
         fields = ('code', 'status', 'testmode', 'email', 'phone', 'locale', 'payment_provider', 'fees', 'comment', 'sales_channel',
                   'invoice_address', 'positions', 'checkin_attention', 'payment_info', 'payment_date', 'consume_carts',
-                  'force', 'send_email', 'simulate', 'customer', 'custom_followup_at')
+                  'force', 'send_email', 'simulate', 'customer', 'custom_followup_at', 'require_approval')
 
     def validate_payment_provider(self, pp):
         if pp is None:
@@ -1041,6 +961,8 @@ class OrderCreateSerializer(I18nAwareModelSerializer):
         force = validated_data.pop('force', False)
         simulate = validated_data.pop('simulate', False)
         self._send_mail = validated_data.pop('send_email', False)
+        if self._send_mail is None:
+            self._send_mail = validated_data.get('sales_channel') in self.context['event'].settings.mail_sales_channel_placed_paid
 
         if 'invoice_address' in validated_data:
             iadata = validated_data.pop('invoice_address')
@@ -1219,6 +1141,8 @@ class OrderCreateSerializer(I18nAwareModelSerializer):
             order.set_expires(subevents=[p.get('subevent') for p in positions_data])
             order.meta_info = "{}"
             order.total = Decimal('0.00')
+            if validated_data.get('require_approval') is not None:
+                order.require_approval = validated_data['require_approval']
             if simulate:
                 order = WrappedModel(order)
                 order.last_modified = now()
@@ -1356,14 +1280,18 @@ class OrderCreateSerializer(I18nAwareModelSerializer):
                     f.order = order._wrapped if simulate else order
                     f._calculate_tax()
                     fees.append(f)
-                    if not simulate:
+                    if simulate:
+                        f.id = 0
+                    else:
                         f.save()
             else:
                 f = OrderFee(**fee_data)
                 f.order = order._wrapped if simulate else order
                 f._calculate_tax()
                 fees.append(f)
-                if not simulate:
+                if simulate:
+                    f.id = 0
+                else:
                     f.save()
 
         order.total += sum([f.value for f in fees])

src/pretix/api/serializers/orderchange.py

@@ -0,0 +1,424 @@
+#
+# This file is part of pretix (Community Edition).
+#
+# Copyright (C) 2014-2020 Raphael Michel and contributors
+# Copyright (C) 2020-2021 rami.io GmbH and contributors
+#
+# This program is free software: you can redistribute it and/or modify it under the terms of the GNU Affero General
+# Public License as published by the Free Software Foundation in version 3 of the License.
+#
+# ADDITIONAL TERMS APPLY: Pursuant to Section 7 of the GNU Affero General Public License, additional terms are
+# applicable granting you additional permissions and placing additional restrictions on your usage of this software.
+# Please refer to the pretix LICENSE file to obtain the full terms applicable to this work. If you did not receive
+# this file, see <https://pretix.eu/about/en/license>.
+#
+# This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied
+# warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU Affero General Public License for more
+# details.
+#
+# You should have received a copy of the GNU Affero General Public License along with this program.  If not, see
+# <https://www.gnu.org/licenses/>.
+#
+import logging
+import os
+
+import pycountry
+from django.core.files import File
+from rest_framework import serializers
+from rest_framework.exceptions import ValidationError
+
+from pretix.api.serializers.order import (
+    AnswerCreateSerializer, AnswerSerializer, CompatibleCountryField,
+    OrderPositionCreateSerializer,
+)
+from pretix.base.models import ItemVariation, Order, OrderFee, OrderPosition
+from pretix.base.services.orders import OrderError
+from pretix.base.settings import COUNTRIES_WITH_STATE_IN_ADDRESS
+
+logger = logging.getLogger(__name__)
+
+
+class OrderPositionCreateForExistingOrderSerializer(OrderPositionCreateSerializer):
+    order = serializers.SlugRelatedField(slug_field='code', queryset=Order.objects.none(), required=True, allow_null=False)
+    answers = AnswerCreateSerializer(many=True, required=False)
+    addon_to = serializers.IntegerField(required=False, allow_null=True)
+    secret = serializers.CharField(required=False)
+    attendee_name = serializers.CharField(required=False, allow_null=True)
+    seat = serializers.CharField(required=False, allow_null=True)
+    price = serializers.DecimalField(required=False, allow_null=True, decimal_places=2,
+                                     max_digits=10)
+    country = CompatibleCountryField(source='*')
+
+    class Meta:
+        model = OrderPosition
+        fields = ('order', 'item', 'variation', 'price', 'attendee_name', 'attendee_name_parts', 'attendee_email',
+                  'company', 'street', 'zipcode', 'city', 'country', 'state',
+                  'secret', 'addon_to', 'subevent', 'answers', 'seat')
+
+    def __init__(self, *args, **kwargs):
+        super().__init__(*args, **kwargs)
+        if not self.context:
+            return
+        self.fields['order'].queryset = self.context['event'].orders.all()
+        self.fields['item'].queryset = self.context['event'].items.all()
+        self.fields['subevent'].queryset = self.context['event'].subevents.all()
+        self.fields['seat'].queryset = self.context['event'].seats.all()
+        self.fields['variation'].queryset = ItemVariation.objects.filter(item__event=self.context['event'])
+        if 'order' in self.context:
+            del self.fields['order']
+
+    def validate(self, data):
+        data = super().validate(data)
+        if data.get('addon_to'):
+            try:
+                data['addon_to'] = data['order'].positions.get(positionid=data['addon_to'])
+            except OrderPosition.DoesNotExist:
+                raise ValidationError({
+                    'addon_to': ['addon_to refers to an unknown position ID for this order.']
+                })
+        return data
+
+    def create(self, validated_data):
+        ocm = self.context['ocm']
+
+        try:
+            ocm.add_position(
+                item=validated_data['item'],
+                variation=validated_data.get('variation'),
+                price=validated_data.get('price'),
+                addon_to=validated_data.get('addon_to'),
+                subevent=validated_data.get('subevent'),
+                seat=validated_data.get('seat'),
+            )
+            if self.context.get('commit', True):
+                ocm.commit()
+                return validated_data['order'].positions.order_by('-positionid').first()
+            else:
+                return OrderPosition()  # fake to appease DRF
+        except OrderError as e:
+            raise ValidationError(str(e))
+
+
+class OrderPositionInfoPatchSerializer(serializers.ModelSerializer):
+    answers = AnswerSerializer(many=True)
+    country = CompatibleCountryField(source='*')
+    attendee_name = serializers.CharField(required=False)
+
+    class Meta:
+        model = OrderPosition
+        fields = (
+            'attendee_name', 'attendee_name_parts', 'company', 'street', 'zipcode', 'city', 'country',
+            'state', 'attendee_email', 'answers',
+        )
+
+    def validate(self, data):
+        if data.get('attendee_name') and data.get('attendee_name_parts'):
+            raise ValidationError(
+                {'attendee_name': ['Do not specify attendee_name if you specified attendee_name_parts.']}
+            )
+        if data.get('attendee_name_parts') and '_scheme' not in data.get('attendee_name_parts'):
+            data['attendee_name_parts']['_scheme'] = self.context['request'].event.settings.name_scheme
+
+        if data.get('country'):
+            if not pycountry.countries.get(alpha_2=data.get('country').code):
+                raise ValidationError(
+                    {'country': ['Invalid country code.']}
+                )
+
+        if data.get('state'):
+            cc = str(data.get('country') or self.instance.country or '')
+            if cc not in COUNTRIES_WITH_STATE_IN_ADDRESS:
+                raise ValidationError(
+                    {'state': ['States are not supported in country "{}".'.format(cc)]}
+                )
+            if not pycountry.subdivisions.get(code=cc + '-' + data.get('state')):
+                raise ValidationError(
+                    {'state': ['"{}" is not a known subdivision of the country "{}".'.format(data.get('state'), cc)]}
+                )
+        return data
+
+    def update(self, instance, validated_data):
+        answers_data = validated_data.pop('answers', None)
+
+        name = validated_data.pop('attendee_name', '')
+        if name and not validated_data.get('attendee_name_parts'):
+            validated_data['attendee_name_parts'] = {
+                '_legacy': name
+            }
+
+        for attr, value in validated_data.items():
+            if attr in self.fields:
+                setattr(instance, attr, value)
+
+        instance.save(update_fields=list(validated_data.keys()))
+
+        if answers_data is not None:
+            qs_seen = set()
+            answercache = {
+                a.question_id: a for a in instance.answers.all()
+            }
+            for answ_data in answers_data:
+                options = answ_data.pop('options', [])
+                if answ_data['question'].pk in qs_seen:
+                    raise ValidationError(f'Question {answ_data["question"]} was sent twice.')
+                if answ_data['question'].pk in answercache:
+                    a = answercache[answ_data['question'].pk]
+                    if isinstance(answ_data['answer'], File):
+                        a.file.save(answ_data['answer'].name, answ_data['answer'], save=False)
+                        a.answer = 'file://' + a.file.name
+                    elif a.answer.startswith('file://') and answ_data['answer'] == "file:keep":
+                        pass  # keep current file
+                    else:
+                        for attr, value in answ_data.items():
+                            setattr(a, attr, value)
+                    a.save()
+                else:
+                    if isinstance(answ_data['answer'], File):
+                        an = answ_data.pop('answer')
+                        a = instance.answers.create(**answ_data, answer='')
+                        a.file.save(os.path.basename(an.name), an, save=False)
+                        a.answer = 'file://' + a.file.name
+                        a.save()
+                    else:
+                        a = instance.answers.create(**answ_data)
+                a.options.set(options)
+                qs_seen.add(a.question_id)
+            for qid, a in answercache.items():
+                if qid not in qs_seen:
+                    a.delete()
+
+        return instance
+
+
+class OrderPositionChangeSerializer(serializers.ModelSerializer):
+    seat = serializers.CharField(source='seat.seat_guid', allow_null=True, required=False)
+
+    class Meta:
+        model = OrderPosition
+        fields = (
+            'item', 'variation', 'subevent', 'seat', 'price', 'tax_rule',
+        )
+
+    def __init__(self, *args, **kwargs):
+        super().__init__(*args, **kwargs)
+        if not self.context:
+            return
+        self.fields['item'].queryset = self.context['event'].items.all()
+        self.fields['subevent'].queryset = self.context['event'].subevents.all()
+        self.fields['tax_rule'].queryset = self.context['event'].tax_rules.all()
+        if kwargs.get('partial'):
+            for k, v in self.fields.items():
+                self.fields[k].required = False
+
+    def validate_item(self, item):
+        if item.event != self.context['event']:
+            raise ValidationError(
+                'The specified item does not belong to this event.'
+            )
+        return item
+
+    def validate_subevent(self, subevent):
+        if self.context['event'].has_subevents:
+            if not subevent:
+                raise ValidationError(
+                    'You need to set a subevent.'
+                )
+            if subevent.event != self.context['event']:
+                raise ValidationError(
+                    'The specified subevent does not belong to this event.'
+                )
+        elif subevent:
+            raise ValidationError(
+                'You cannot set a subevent for this event.'
+            )
+        return subevent
+
+    def validate(self, data, instance=None):
+        instance = instance or self.instance
+        if instance is None:
+            return data  # needs to be done later
+        if data.get('item', instance.item):
+            if data.get('item', instance.item).has_variations:
+                if not data.get('variation', instance.variation):
+                    raise ValidationError({'variation': ['You should specify a variation for this item.']})
+                else:
+                    if data.get('variation', instance.variation).item != data.get('item', instance.item):
+                        raise ValidationError(
+                            {'variation': ['The specified variation does not belong to the specified item.']}
+                        )
+            elif data.get('variation', instance.variation):
+                raise ValidationError(
+                    {'variation': ['You cannot specify a variation for this item.']}
+                )
+
+        return data
+
+    def update(self, instance, validated_data):
+        ocm = self.context['ocm']
+        current_seat = {'seat_guid': instance.seat.seat_guid} if instance.seat else None
+        item = validated_data.get('item', instance.item)
+        variation = validated_data.get('variation', instance.variation)
+        subevent = validated_data.get('subevent', instance.subevent)
+        price = validated_data.get('price', instance.price)
+        seat = validated_data.get('seat', current_seat)
+        tax_rule = validated_data.get('tax_rule', instance.tax_rule)
+
+        change_item = None
+        if item != instance.item or variation != instance.variation:
+            change_item = (item, variation)
+
+        change_subevent = None
+        if self.context['event'].has_subevents and subevent != instance.subevent:
+            change_subevent = (subevent,)
+
+        try:
+            if change_item is not None and change_subevent is not None:
+                ocm.change_item_and_subevent(instance, *change_item, *change_subevent)
+            elif change_item is not None:
+                ocm.change_item(instance, *change_item)
+            elif change_subevent is not None:
+                ocm.change_subevent(instance, *change_subevent)
+
+            if seat != current_seat or change_subevent:
+                ocm.change_seat(instance, seat['seat_guid'] if seat else None)
+
+            if price != instance.price:
+                ocm.change_price(instance, price)
+
+            if tax_rule != instance.tax_rule:
+                ocm.change_tax_rule(instance, tax_rule)
+
+            if self.context.get('commit', True):
+                ocm.commit()
+                instance.refresh_from_db()
+        except OrderError as e:
+            raise ValidationError(str(e))
+        return instance
+
+
+class PatchPositionSerializer(serializers.Serializer):
+    position = serializers.PrimaryKeyRelatedField(queryset=OrderPosition.all.none())
+
+    def validate_position(self, value):
+        self.fields['body'].instance = value  # hack around DRFs validation order
+        return value
+
+    def validate(self, data):
+        OrderPositionChangeSerializer(context=self.context, partial=True).validate(data['body'], data['position'])
+        return data
+
+    def __init__(self, *args, **kwargs):
+        super().__init__(*args, **kwargs)
+        self.fields['position'].queryset = self.context['order'].positions.all()
+        self.fields['body'] = OrderPositionChangeSerializer(context=self.context, partial=True)
+
+
+class SelectPositionSerializer(serializers.Serializer):
+    position = serializers.PrimaryKeyRelatedField(queryset=OrderPosition.all.none())
+
+    def __init__(self, *args, **kwargs):
+        super().__init__(*args, **kwargs)
+        self.fields['position'].queryset = self.context['order'].positions.all()
+
+
+class OrderFeeChangeSerializer(serializers.ModelSerializer):
+
+    class Meta:
+        model = OrderFee
+        fields = (
+            'value',
+        )
+
+    def update(self, instance, validated_data):
+        ocm = self.context['ocm']
+        value = validated_data.get('value', instance.value)
+
+        try:
+            if value != instance.value:
+                ocm.change_fee(instance, value)
+
+            if self.context.get('commit', True):
+                ocm.commit()
+                instance.refresh_from_db()
+        except OrderError as e:
+            raise ValidationError(str(e))
+        return instance
+
+
+class PatchFeeSerializer(serializers.Serializer):
+    fee = serializers.PrimaryKeyRelatedField(queryset=OrderFee.all.none())
+
+    def __init__(self, *args, **kwargs):
+        super().__init__(*args, **kwargs)
+        self.fields['fee'].queryset = self.context['order'].fees.all()
+        self.fields['body'] = OrderFeeChangeSerializer(context=self.context)
+
+
+class SelectFeeSerializer(serializers.Serializer):
+    fee = serializers.PrimaryKeyRelatedField(queryset=OrderFee.all.none())
+
+    def __init__(self, *args, **kwargs):
+        super().__init__(*args, **kwargs)
+        if not self.context:
+            return
+        self.fields['fee'].queryset = self.context['order'].fees.all()
+
+
+class OrderChangeOperationSerializer(serializers.Serializer):
+    send_email = serializers.BooleanField(default=False, required=False)
+    reissue_invoice = serializers.BooleanField(default=True, required=False)
+    recalculate_taxes = serializers.ChoiceField(default=None, allow_null=True, required=False, choices=[
+        ('keep_net', 'keep_net'),
+        ('keep_gross', 'keep_gross'),
+    ])
+
+    def __init__(self, *args, **kwargs):
+        super().__init__(self, *args, **kwargs)
+        self.fields['patch_positions'] = PatchPositionSerializer(
+            many=True, required=False, context=self.context
+        )
+        self.fields['cancel_positions'] = SelectPositionSerializer(
+            many=True, required=False, context=self.context
+        )
+        self.fields['create_positions'] = OrderPositionCreateForExistingOrderSerializer(
+            many=True, required=False, context=self.context
+        )
+        self.fields['split_positions'] = SelectPositionSerializer(
+            many=True, required=False, context=self.context
+        )
+        self.fields['patch_fees'] = PatchFeeSerializer(
+            many=True, required=False, context=self.context
+        )
+        self.fields['cancel_fees'] = SelectFeeSerializer(
+            many=True, required=False, context=self.context
+        )
+
+    def validate(self, data):
+        seen_positions = set()
+        for d in data.get('patch_positions', []):
+            print(d, seen_positions)
+            if d['position'] in seen_positions:
+                raise ValidationError({'patch_positions': ['You have specified the same object twice.']})
+            seen_positions.add(d['position'])
+        seen_positions = set()
+        for d in data.get('cancel_positions', []):
+            if d['position'] in seen_positions:
+                raise ValidationError({'cancel_positions': ['You have specified the same object twice.']})
+            seen_positions.add(d['position'])
+        seen_positions = set()
+        for d in data.get('split_positions', []):
+            if d['position'] in seen_positions:
+                raise ValidationError({'split_positions': ['You have specified the same object twice.']})
+            seen_positions.add(d['position'])
+        seen_fees = set()
+        for d in data.get('patch_fees', []):
+            if d['fee'] in seen_fees:
+                raise ValidationError({'patch_fees': ['You have specified the same object twice.']})
+            seen_positions.add(d['fee'])
+        seen_fees = set()
+        for d in data.get('cancel_fees', []):
+            if d['fee'] in seen_fees:
+                raise ValidationError({'cancel_fees': ['You have specified the same object twice.']})
+            seen_positions.add(d['fee'])
+
+        return data

src/pretix/api/views/device.py

@@ -42,6 +42,7 @@ class InitializationRequestSerializer(serializers.Serializer):
     hardware_model = serializers.CharField(max_length=190)
     software_brand = serializers.CharField(max_length=190)
     software_version = serializers.CharField(max_length=190)
+    info = serializers.JSONField(required=False, allow_null=True)
 
 
 class UpdateRequestSerializer(serializers.Serializer):
@@ -49,6 +50,7 @@ class UpdateRequestSerializer(serializers.Serializer):
     hardware_model = serializers.CharField(max_length=190)
     software_brand = serializers.CharField(max_length=190)
     software_version = serializers.CharField(max_length=190)
+    info = serializers.JSONField(required=False, allow_null=True)
 
 
 class GateSerializer(serializers.ModelSerializer):
@@ -94,6 +96,7 @@ class InitializeView(APIView):
         device.hardware_model = serializer.validated_data.get('hardware_model')
         device.software_brand = serializer.validated_data.get('software_brand')
         device.software_version = serializer.validated_data.get('software_version')
+        device.info = serializer.validated_data.get('info')
         device.api_token = generate_api_token()
         device.save()
 
@@ -114,6 +117,7 @@ class UpdateView(APIView):
         device.hardware_model = serializer.validated_data.get('hardware_model')
         device.software_brand = serializer.validated_data.get('software_brand')
         device.software_version = serializer.validated_data.get('software_version')
+        device.info = serializer.validated_data.get('info')
         device.save()
         device.log_action('pretix.device.updated', data=serializer.validated_data, auth=device)
 

src/pretix/api/views/exporters.py

@@ -132,7 +132,7 @@ class EventExportersViewSet(ExportersMixin, viewsets.ViewSet):
     def exporters(self):
         exporters = []
         responses = register_data_exporters.send(self.request.event)
-        for ex in sorted([response(self.request.event, self.request.organizer) for r, response in responses], key=lambda ex: str(ex.verbose_name)):
+        for ex in sorted([response(self.request.event, self.request.organizer) for r, response in responses if response], key=lambda ex: str(ex.verbose_name)):
             ex._serializer = JobRunSerializer(exporter=ex)
             exporters.append(ex)
         return exporters
@@ -147,7 +147,11 @@ class OrganizerExportersViewSet(ExportersMixin, viewsets.ViewSet):
     @cached_property
     def exporters(self):
         exporters = []
-        events = (self.request.auth or self.request.user).get_events_with_permission('can_view_orders', request=self.request).filter(
+        if isinstance(self.request.auth, (Device, TeamAPIToken)):
+            perm_holder = self.request.auth
+        else:
+            perm_holder = self.request.user
+        events = perm_holder.get_events_with_permission('can_view_orders', request=self.request).filter(
             organizer=self.request.organizer
         )
         responses = register_multievent_data_exporters.send(self.request.organizer)
@@ -157,8 +161,12 @@ class OrganizerExportersViewSet(ExportersMixin, viewsets.ViewSet):
         return exporters
 
     def get_serializer_kwargs(self):
+        if isinstance(self.request.auth, (Device, TeamAPIToken)):
+            perm_holder = self.request.auth
+        else:
+            perm_holder = self.request.user
         return {
-            'events': self.request.auth.get_events_with_permission('can_view_orders', request=self.request).filter(
+            'events': perm_holder.get_events_with_permission('can_view_orders', request=self.request).filter(
                 organizer=self.request.organizer
             )
         }

src/pretix/api/views/order.py

@@ -36,7 +36,7 @@ from django.utils.translation import gettext as _
 from django_filters.rest_framework import DjangoFilterBackend, FilterSet
 from django_scopes import scopes_disabled
 from PIL import Image
-from rest_framework import mixins, serializers, status, viewsets
+from rest_framework import serializers, status, viewsets
 from rest_framework.decorators import action
 from rest_framework.exceptions import (
     APIException, NotFound, PermissionDenied, ValidationError,
@@ -53,6 +53,12 @@ from pretix.api.serializers.order import (
     PriceCalcSerializer, RevokedTicketSecretSerializer,
     SimulatedOrderSerializer,
 )
+from pretix.api.serializers.orderchange import (
+    OrderChangeOperationSerializer, OrderFeeChangeSerializer,
+    OrderPositionChangeSerializer,
+    OrderPositionCreateForExistingOrderSerializer,
+    OrderPositionInfoPatchSerializer,
+)
 from pretix.base.i18n import language
 from pretix.base.models import (
     CachedCombinedTicket, CachedTicket, Checkin, Device, Event, Invoice,
@@ -144,7 +150,8 @@ with scopes_disabled():
             matching_positions = OrderPosition.objects.filter(
                 Q(order=OuterRef('pk')) & Q(
                     Q(attendee_name_cached__icontains=u) | Q(attendee_email__icontains=u)
-                    | Q(secret__istartswith=u) | Q(voucher__code__icontains=u)
+                    | Q(secret__istartswith=u)
+                    # | Q(voucher__code__icontains=u)  # temporarily removed since it caused bad query performance on postgres
                 )
             ).values('id')
 
@@ -338,6 +345,7 @@ class OrderViewSet(viewsets.ModelViewSet):
     @action(detail=True, methods=['POST'])
     def mark_canceled(self, request, **kwargs):
         send_mail = request.data.get('send_email', True)
+        comment = request.data.get('comment', None)
         cancellation_fee = request.data.get('cancellation_fee', None)
         if cancellation_fee:
             try:
@@ -360,6 +368,7 @@ class OrderViewSet(viewsets.ModelViewSet):
                 device=request.auth if isinstance(request.auth, Device) else None,
                 oauth_application=request.auth.application if isinstance(request.auth, OAuthAccessToken) else None,
                 send_mail=send_mail,
+                email_comment=comment,
                 cancellation_fee=cancellation_fee
             )
         except OrderError as e:
@@ -644,9 +653,13 @@ class OrderViewSet(viewsets.ModelViewSet):
             if send_mail:
                 free_flow = (
                     payment and order.total == Decimal('0.00') and order.status == Order.STATUS_PAID and
-                    not order.require_approval and payment.provider == "free"
+                    not order.require_approval and payment.provider in ("free", "boxoffice")
                 )
-                if free_flow:
+                if order.require_approval:
+                    email_template = request.event.settings.mail_text_order_placed_require_approval
+                    log_entry = 'pretix.event.order.email.order_placed_require_approval'
+                    email_attendees = False
+                elif free_flow:
                     email_template = request.event.settings.mail_text_order_free
                     log_entry = 'pretix.event.order.email.order_free'
                     email_attendees = request.event.settings.mail_send_order_free_attendee
@@ -659,12 +672,13 @@ class OrderViewSet(viewsets.ModelViewSet):
 
                 _order_placed_email(
                     request.event, order, payment.payment_provider if payment else None, email_template,
-                    log_entry, invoice, payment
+                    log_entry, invoice, payment, is_free=free_flow
                 )
                 if email_attendees:
                     for p in order.positions.all():
                         if p.addon_to_id is None and p.attendee_email and p.attendee_email != order.email:
-                            _order_placed_email_attendee(request.event, order, p, email_attendees_template, log_entry)
+                            _order_placed_email_attendee(request.event, order, p, email_attendees_template, log_entry,
+                                                         is_free=free_flow)
 
                 if not free_flow and order.status == Order.STATUS_PAID and payment:
                     payment._send_paid_mail(invoice, None, '')
@@ -777,6 +791,79 @@ class OrderViewSet(viewsets.ModelViewSet):
         with transaction.atomic():
             self.get_object().gracefully_delete(user=self.request.user if self.request.user.is_authenticated else None, auth=self.request.auth)
 
+    @action(detail=True, methods=['POST'])
+    def change(self, request, **kwargs):
+        order = self.get_object()
+
+        serializer = OrderChangeOperationSerializer(
+            context={'order': order, **self.get_serializer_context()},
+            data=request.data,
+        )
+        serializer.is_valid(raise_exception=True)
+
+        try:
+            ocm = OrderChangeManager(
+                order=order,
+                user=self.request.user if self.request.user.is_authenticated else None,
+                auth=request.auth,
+                notify=serializer.validated_data.get('send_email', False),
+                reissue_invoice=serializer.validated_data.get('reissue_invoice', True),
+            )
+
+            canceled_positions = set()
+            for r in serializer.validated_data.get('cancel_positions', []):
+                ocm.cancel(r['position'])
+                canceled_positions.add(r['position'])
+
+            for r in serializer.validated_data.get('patch_positions', []):
+                if r['position'] in canceled_positions:
+                    continue
+                pos_serializer = OrderPositionChangeSerializer(
+                    context={'ocm': ocm, 'commit': False, 'event': request.event, **self.get_serializer_context()},
+                    partial=True,
+                )
+                pos_serializer.update(r['position'], r['body'])
+
+            for r in serializer.validated_data.get('split_positions', []):
+                if r['position'] in canceled_positions:
+                    continue
+                ocm.split(r['position'])
+
+            for r in serializer.validated_data.get('create_positions', []):
+                pos_serializer = OrderPositionCreateForExistingOrderSerializer(
+                    context={'ocm': ocm, 'commit': False, 'event': request.event, **self.get_serializer_context()},
+                )
+                pos_serializer.create(r)
+
+            canceled_fees = set()
+            for r in serializer.validated_data.get('cancel_fees', []):
+                ocm.cancel_fee(r['fee'])
+                canceled_fees.add(r['fee'])
+
+            for r in serializer.validated_data.get('patch_fees', []):
+                if r['fee'] in canceled_fees:
+                    continue
+                pos_serializer = OrderFeeChangeSerializer(
+                    context={'ocm': ocm, 'commit': False, 'event': request.event, **self.get_serializer_context()},
+                )
+                pos_serializer.update(r['fee'], r['body'])
+
+            if serializer.validated_data.get('recalculate_taxes') == 'keep_net':
+                ocm.recalculate_taxes(keep='net')
+            elif serializer.validated_data.get('recalculate_taxes') == 'keep_gross':
+                ocm.recalculate_taxes(keep='gross')
+
+            ocm.commit()
+        except OrderError as e:
+            raise ValidationError(str(e))
+
+        order.refresh_from_db()
+        serializer = OrderSerializer(
+            instance=order,
+            context=self.get_serializer_context(),
+        )
+        return Response(serializer.data)
+
 
 with scopes_disabled():
     class OrderPositionFilter(FilterSet):
@@ -818,7 +905,7 @@ with scopes_disabled():
             }
 
 
-class OrderPositionViewSet(mixins.DestroyModelMixin, mixins.UpdateModelMixin, viewsets.ReadOnlyModelViewSet):
+class OrderPositionViewSet(viewsets.ModelViewSet):
     serializer_class = OrderPositionSerializer
     queryset = OrderPosition.all.none()
     filter_backends = (DjangoFilterBackend, OrderingFilter)
@@ -1055,6 +1142,25 @@ class OrderPositionViewSet(mixins.DestroyModelMixin, mixins.UpdateModelMixin, vi
                 )
                 return resp
 
+    @action(detail=True, methods=['POST'])
+    def regenerate_secrets(self, request, **kwargs):
+        instance = self.get_object()
+        try:
+            ocm = OrderChangeManager(
+                instance.order,
+                user=self.request.user if self.request.user.is_authenticated else None,
+                auth=self.request.auth,
+                notify=False,
+                reissue_invoice=False,
+            )
+            ocm.regenerate_secret(instance)
+            ocm.commit()
+        except OrderError as e:
+            raise ValidationError(str(e))
+        except Quota.QuotaExceededException as e:
+            raise ValidationError(str(e))
+        return self.retrieve(request, [], **kwargs)
+
     def perform_destroy(self, instance):
         try:
             ocm = OrderChangeManager(
@@ -1070,18 +1176,33 @@ class OrderPositionViewSet(mixins.DestroyModelMixin, mixins.UpdateModelMixin, vi
         except Quota.QuotaExceededException as e:
             raise ValidationError(str(e))
 
-    def update(self, request, *args, **kwargs):
-        partial = kwargs.get('partial', False)
-        if not partial:
-            return Response(
-                {"detail": "Method \"PUT\" not allowed."},
-                status=status.HTTP_405_METHOD_NOT_ALLOWED,
-            )
-        return super().update(request, *args, **kwargs)
-
-    def perform_update(self, serializer):
+    def create(self, request, *args, **kwargs):
         with transaction.atomic():
-            old_data = self.get_serializer_class()(instance=serializer.instance, context=self.get_serializer_context()).data
+            serializer = OrderPositionCreateForExistingOrderSerializer(
+                data=request.data,
+                context=self.get_serializer_context(),
+            )
+            serializer.is_valid(raise_exception=True)
+            order = serializer.validated_data['order']
+            ocm = OrderChangeManager(
+                order=order,
+                user=self.request.user if self.request.user.is_authenticated else None,
+                auth=request.auth,
+                notify=False,
+                reissue_invoice=False,
+            )
+            serializer.context['ocm'] = ocm
+            serializer.save()
+
+            # Fields that can be easily patched after the position was added
+            old_data = OrderPositionInfoPatchSerializer(instance=serializer.instance, context=self.get_serializer_context()).data
+            serializer = OrderPositionInfoPatchSerializer(
+                instance=serializer.instance,
+                context=self.get_serializer_context(),
+                partial=True,
+                data=request.data
+            )
+            serializer.is_valid(raise_exception=True)
             serializer.save()
             new_data = serializer.data
 
@@ -1104,9 +1225,77 @@ class OrderPositionViewSet(mixins.DestroyModelMixin, mixins.UpdateModelMixin, vi
                         ]
                     }
                 )
+                tickets.invalidate_cache.apply_async(
+                    kwargs={'event': serializer.instance.order.event.pk, 'order': serializer.instance.order.pk})
+                order_modified.send(sender=serializer.instance.order.event, order=serializer.instance.order)
+        return Response(
+            OrderPositionSerializer(serializer.instance, context=self.get_serializer_context()).data,
+            status=status.HTTP_201_CREATED,
+        )
 
-        tickets.invalidate_cache.apply_async(kwargs={'event': serializer.instance.order.event.pk, 'order': serializer.instance.order.pk})
-        order_modified.send(sender=serializer.instance.order.event, order=serializer.instance.order)
+    def update(self, request, *args, **kwargs):
+        partial = kwargs.get('partial', False)
+        if not partial:
+            return Response(
+                {"detail": "Method \"PUT\" not allowed."},
+                status=status.HTTP_405_METHOD_NOT_ALLOWED,
+            )
+
+        with transaction.atomic():
+            instance = self.get_object()
+            ocm = OrderChangeManager(
+                order=instance.order,
+                user=self.request.user if self.request.user.is_authenticated else None,
+                auth=request.auth,
+                notify=False,
+                reissue_invoice=False,
+            )
+
+            # Field that need to go through OrderChangeManager
+            serializer = OrderPositionChangeSerializer(
+                instance=instance,
+                context={'ocm': ocm, **self.get_serializer_context()},
+                partial=True,
+                data=request.data
+            )
+            serializer.is_valid(raise_exception=True)
+            serializer.save()
+
+            # Fields that can be easily patched
+            old_data = OrderPositionInfoPatchSerializer(instance=instance, context=self.get_serializer_context()).data
+            serializer = OrderPositionInfoPatchSerializer(
+                instance=instance,
+                context=self.get_serializer_context(),
+                partial=True,
+                data=request.data
+            )
+            serializer.is_valid(raise_exception=True)
+            serializer.save()
+            new_data = serializer.data
+
+            if old_data != new_data:
+                log_data = self.request.data
+                if 'answers' in log_data:
+                    for a in new_data['answers']:
+                        log_data[f'question_{a["question"]}'] = a["answer"]
+                    log_data.pop('answers', None)
+                serializer.instance.order.log_action(
+                    'pretix.event.order.modified',
+                    user=self.request.user,
+                    auth=self.request.auth,
+                    data={
+                        'data': [
+                            dict(
+                                position=serializer.instance.pk,
+                                **log_data
+                            )
+                        ]
+                    }
+                )
+                tickets.invalidate_cache.apply_async(kwargs={'event': serializer.instance.order.event.pk, 'order': serializer.instance.order.pk})
+                order_modified.send(sender=serializer.instance.order.event, order=serializer.instance.order)
+
+        return Response(self.get_serializer_class()(instance=serializer.instance, context=self.get_serializer_context()).data)
 
 
 class PaymentViewSet(CreateModelMixin, viewsets.ReadOnlyModelViewSet):

src/pretix/base/auth.py

@@ -94,6 +94,9 @@ class BaseAuthBackend:
         This method will be called after the user filled in the login form. ``request`` will contain
         the current request and ``form_data`` the input for the form fields defined in ``login_form_fields``.
         You are expected to either return a ``User`` object (if login was successful) or ``None``.
+
+        You are expected to either return a ``User`` object (if login was successful) or ``None``. You should
+        obtain this user object using ``User.objects.get_or_create_for_backend``.
         """
         return
 
@@ -104,7 +107,9 @@ class BaseAuthBackend:
         reverse proxy, you can directly return a ``User`` object that will be logged in.
 
         ``request`` will contain the current request.
-        You are expected to either return a ``User`` object (if login was successful) or ``None``.
+
+        You are expected to either return a ``User`` object (if login was successful) or ``None``. You should
+        obtain this user object using ``User.objects.get_or_create_for_backend``.
         """
         return
 

src/pretix/base/email.py

@@ -33,6 +33,7 @@ from django.core.mail.backends.smtp import EmailBackend
 from django.db.models import Count
 from django.dispatch import receiver
 from django.template.loader import get_template
+from django.utils.formats import date_format
 from django.utils.timezone import now
 from django.utils.translation import (
     get_language, gettext_lazy as _, pgettext_lazy,
@@ -164,9 +165,20 @@ class TemplateBasedMailRenderer(BaseHTMLMailRenderer):
                 has_addons=Count('addons')
             ))
             htmlctx['cart'] = [(k, list(v)) for k, v in groupby(
-                positions, key=lambda op: (
-                    op.item, op.variation, op.subevent, op.attendee_name,
-                    (op.pk if op.addon_to_id else None), (op.pk if op.has_addons else None)
+                sorted(
+                    positions,
+                    key=lambda op: (
+                        (op.addon_to.positionid if op.addon_to_id else op.positionid),
+                        op.positionid
+                    )
+                ),
+                key=lambda op: (
+                    op.item,
+                    op.variation,
+                    op.subevent,
+                    op.attendee_name,
+                    op.addon_to_id,
+                    (op.pk if op.has_addons else None)
                 )
             )]
 
@@ -298,7 +310,11 @@ def get_email_context(**kwargs):
             val = [val]
         for v in val:
             if all(rp in kwargs for rp in v.required_context):
-                ctx[v.identifier] = v.render(kwargs)
+                try:
+                    ctx[v.identifier] = v.render(kwargs)
+                except:
+                    ctx[v.identifier] = '(error)'
+                    logger.exception(f'Failed to process email placeholder {v.identifier}.')
     return ctx
 
 
@@ -453,6 +469,15 @@ def base_placeholders(sender, **kwargs):
                 }
             ),
         ),
+        SimpleFunctionalMailTextPlaceholder(
+            'event_location', ['event_or_subevent'], lambda event_or_subevent: str(event_or_subevent.location or ''),
+            lambda event: str(event.location or ''),
+        ),
+        SimpleFunctionalMailTextPlaceholder(
+            'event_admission_time', ['event_or_subevent'],
+            lambda event_or_subevent: date_format(event_or_subevent.date_admission, 'TIME_FORMAT') if event_or_subevent.date_admission else '',
+            lambda event: date_format(event.date_admission, 'TIME_FORMAT') if event.date_admission else '',
+        ),
         SimpleFunctionalMailTextPlaceholder(
             'subevent', ['waiting_list_entry', 'event'],
             lambda waiting_list_entry, event: str(waiting_list_entry.subevent or event),
@@ -622,6 +647,10 @@ def base_placeholders(sender, **kwargs):
             'meta_%s' % k, ['event'], lambda event, k=k: event.meta_data[k],
             v
         ))
+        ph.append(SimpleFunctionalMailTextPlaceholder(
+            'meta_%s' % k, ['event_or_subevent'], lambda event_or_subevent, k=k: event_or_subevent.meta_data[k],
+            v
+        ))
 
     return ph
 

src/pretix/base/exporter.py

@@ -33,7 +33,6 @@
 # License for the specific language governing permissions and limitations under the License.
 
 import io
-import re
 import tempfile
 from collections import OrderedDict, namedtuple
 from decimal import Decimal
@@ -46,26 +45,13 @@ from django.conf import settings
 from django.db.models import QuerySet
 from django.utils.formats import localize
 from django.utils.translation import gettext, gettext_lazy as _
-from openpyxl import Workbook
-from openpyxl.cell.cell import ILLEGAL_CHARACTERS_RE, KNOWN_TYPES, Cell
 
 from pretix.base.models import Event
+from pretix.helpers.safe_openpyxl import (  # NOQA: backwards compatibility for plugins using excel_safe
+    SafeWorkbook, remove_invalid_excel_chars as excel_safe,
+)
 
-
-def excel_safe(val):
-    if isinstance(val, Cell):
-        return val
-
-    if not isinstance(val, KNOWN_TYPES):
-        val = str(val)
-
-    if isinstance(val, bytes):
-        val = val.decode("utf-8", errors="ignore")
-
-    if isinstance(val, str):
-        val = re.sub(ILLEGAL_CHARACTERS_RE, '', val)
-
-    return val
+__ = excel_safe  # just so the compatbility import above is "used" and doesn't get removed by linter
 
 
 class BaseExporter:
@@ -228,7 +214,7 @@ class ListExporter(BaseExporter):
         pass
 
     def _render_xlsx(self, form_data, output_file=None):
-        wb = Workbook(write_only=True)
+        wb = SafeWorkbook(write_only=True)
         ws = wb.create_sheet()
         self.prepare_xlsx_sheet(ws)
         try:
@@ -242,7 +228,7 @@ class ListExporter(BaseExporter):
                 total = line.total
                 continue
             ws.append([
-                excel_safe(val) for val in line
+                val for val in line
             ])
             if total:
                 counter += 1
@@ -347,7 +333,7 @@ class MultiSheetListExporter(ListExporter):
             return self.get_filename() + '.csv', 'text/csv', output.getvalue().encode("utf-8")
 
     def _render_xlsx(self, form_data, output_file=None):
-        wb = Workbook(write_only=True)
+        wb = SafeWorkbook(write_only=True)
         n_sheets = len(self.sheets)
         for i_sheet, (s, l) in enumerate(self.sheets):
             ws = wb.create_sheet(str(l))
@@ -361,8 +347,7 @@ class MultiSheetListExporter(ListExporter):
                     total = line.total
                     continue
                 ws.append([
-                    excel_safe(val)
-                    for val in line
+                    val for val in line
                 ])
                 if total:
                     counter += 1

src/pretix/base/exporters/__init__.py

@@ -21,6 +21,7 @@
 #
 from .answers import *  # noqa
 from .dekodi import *  # noqa
+from .events import *  # noqa
 from .invoices import *  # noqa
 from .json import *  # noqa
 from .mail import *  # noqa

src/pretix/base/exporters/events.py

@@ -0,0 +1,100 @@
+#
+# This file is part of pretix (Community Edition).
+#
+# Copyright (C) 2014-2020 Raphael Michel and contributors
+# Copyright (C) 2020-2021 rami.io GmbH and contributors
+#
+# This program is free software: you can redistribute it and/or modify it under the terms of the GNU Affero General
+# Public License as published by the Free Software Foundation in version 3 of the License.
+#
+# ADDITIONAL TERMS APPLY: Pursuant to Section 7 of the GNU Affero General Public License, additional terms are
+# applicable granting you additional permissions and placing additional restrictions on your usage of this software.
+# Please refer to the pretix LICENSE file to obtain the full terms applicable to this work. If you did not receive
+# this file, see <https://pretix.eu/about/en/license>.
+#
+# This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied
+# warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU Affero General Public License for more
+# details.
+#
+# You should have received a copy of the GNU Affero General Public License along with this program.  If not, see
+# <https://www.gnu.org/licenses/>.
+#
+
+# This file is based on an earlier version of pretix which was released under the Apache License 2.0. The full text of
+# the Apache License 2.0 can be obtained at <http://www.apache.org/licenses/LICENSE-2.0>.
+#
+# This file may have since been changed and any changes are released under the terms of AGPLv3 as described above. A
+# full history of changes and contributors is available at <https://github.com/pretix/pretix>.
+#
+# This file contains Apache-licensed contributions copyrighted by: Tobias Kunze
+#
+# Unless required by applicable law or agreed to in writing, software distributed under the Apache License 2.0 is
+# distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+# License for the specific language governing permissions and limitations under the License.
+
+from django.dispatch import receiver
+from django.utils.formats import date_format
+from django.utils.functional import cached_property
+from django.utils.translation import gettext_lazy as _
+
+from ...control.forms.filter import get_all_payment_providers
+from ..exporter import ListExporter
+from ..signals import register_multievent_data_exporters
+
+
+class EventDataExporter(ListExporter):
+    identifier = 'eventdata'
+    verbose_name = _('Event data')
+
+    @cached_property
+    def providers(self):
+        return dict(get_all_payment_providers())
+
+    def iterate_list(self, form_data):
+        header = [
+            _("Event name"),
+            _("Short form"),
+            _("Shop is live"),
+            _("Event currency"),
+            _("Event start time"),
+            _("Event end time"),
+            _("Admission time"),
+            _("Start of presale"),
+            _("End of presale"),
+            _("Location"),
+            _("Latitude"),
+            _("Longitude"),
+            _("Internal comment"),
+        ]
+        props = list(self.organizer.meta_properties.all())
+        for p in props:
+            header.append(p.name)
+        yield header
+
+        for e in self.events.all():
+            m = e.meta_data
+            yield [
+                str(e.name),
+                e.slug,
+                _('Yes') if e.live else _('No'),
+                e.currency,
+                date_format(e.date_from, 'SHORT_DATETIME_FORMAT'),
+                date_format(e.date_to, 'SHORT_DATETIME_FORMAT') if e.date_to else '',
+                date_format(e.date_admission, 'SHORT_DATETIME_FORMAT') if e.date_admission else '',
+                date_format(e.presale_start, 'SHORT_DATETIME_FORMAT') if e.presale_start else '',
+                date_format(e.presale_end, 'SHORT_DATETIME_FORMAT') if e.presale_end else '',
+                str(e.location),
+                e.geo_lat or '',
+                e.geo_lon or '',
+                e.comment,
+            ] + [
+                m.get(p.name, '') for p in props
+            ]
+
+    def get_filename(self):
+        return '{}_events'.format(self.events.first().organizer.slug)
+
+
+@receiver(register_multievent_data_exporters, dispatch_uid="multiexporter_eventdata")
+def register_multievent_eventdata_exporter(sender, **kwargs):
+    return EventDataExporter

src/pretix/base/exporters/orderlist.py

@@ -573,6 +573,7 @@ class OrderListExporter(MultiSheetListExporter):
             pgettext('address', 'State'),
             _('Voucher'),
             _('Pseudonymization ID'),
+            _('Ticket secret'),
             _('Seat ID'),
             _('Seat name'),
             _('Seat zone'),
@@ -669,6 +670,7 @@ class OrderListExporter(MultiSheetListExporter):
                     op.state or '',
                     op.voucher.code if op.voucher else '',
                     op.pseudonymization_id,
+                    op.secret,
                 ]
 
                 if op.seat:

src/pretix/base/forms/__init__.py

@@ -38,6 +38,7 @@ import i18nfield.forms
 from django import forms
 from django.forms.models import ModelFormMetaclass
 from django.utils.crypto import get_random_string
+from django.utils.translation import gettext_lazy as _
 from formtools.wizard.views import SessionWizardView
 from hierarkey.forms import HierarkeyForm
 
@@ -112,10 +113,13 @@ class SettingsForm(i18nfield.forms.I18nFormMixin, HierarkeyForm):
             if isinstance(f, (RelativeDateTimeField, RelativeDateField)):
                 f.set_event(self.obj)
 
-    def save(self):
+    def _unmask_secret_fields(self):
         for k, v in self.cleaned_data.items():
             if isinstance(self.fields.get(k), SecretKeySettingsField) and self.cleaned_data.get(k) == SECRET_REDACTED:
                 self.cleaned_data[k] = self.initial[k]
+
+    def save(self):
+        self._unmask_secret_fields()
         return super().save()
 
     def clean(self):
@@ -128,6 +132,12 @@ class SettingsForm(i18nfield.forms.I18nFormMixin, HierarkeyForm):
         # at all, it will be considered a changed value and stored. We do not want that, as it makes it very hard to add
         # languages to an organizer/event later on. So we trick it and make sure nothing gets changed in that situation.
         for name, field in self.fields.items():
+            if isinstance(field, SecretKeySettingsField) and d.get(name) == SECRET_REDACTED and not self.initial.get(name):
+                self.add_error(
+                    name,
+                    _('Due to technical reasons you cannot set inputs, that need to be masked (e.g. passwords), to %(value)s.') % {'value': SECRET_REDACTED}
+                )
+
             if isinstance(field, i18nfield.forms.I18nFormField):
                 value = d.get(name)
                 if not value:

src/pretix/base/forms/questions.py

@@ -41,16 +41,16 @@ from io import BytesIO
 import dateutil.parser
 import pycountry
 import pytz
-from babel import Locale
 from django import forms
 from django.conf import settings
 from django.contrib import messages
 from django.core.exceptions import ValidationError
 from django.core.files.uploadedfile import SimpleUploadedFile
-from django.core.validators import MaxValueValidator, MinValueValidator
+from django.core.validators import (
+    MaxValueValidator, MinValueValidator, RegexValidator,
+)
 from django.db.models import QuerySet
 from django.forms import Select, widgets
-from django.utils import translation
 from django.utils.formats import date_format
 from django.utils.html import escape
 from django.utils.safestring import mark_safe
@@ -85,7 +85,9 @@ from pretix.base.templatetags.rich_text import rich_text
 from pretix.control.forms import (
     ExtFileField, ExtValidationMixin, SizeValidationMixin, SplitDateTimeField,
 )
-from pretix.helpers.countries import CachedCountries
+from pretix.helpers.countries import (
+    CachedCountries, get_phone_prefixes_sorted_and_localized,
+)
 from pretix.helpers.escapejson import escapejson_attr
 from pretix.helpers.i18n import get_format_without_seconds
 from pretix.presale.signals import question_form_fields
@@ -187,6 +189,15 @@ class NamePartsFormField(forms.MultiValueField):
         defaults = {
             'widget': self.widget,
             'max_length': kwargs.pop('max_length', None),
+            'validators': [
+                RegexValidator(
+                    # The following characters should never appear in a name anywhere of
+                    # the world. However, they commonly appear in inputs generated by spam
+                    # bots.
+                    r'^[^$€/%§{}<>~]*$',
+                    message=_('Please do not use special characters in names.')
+                )
+            ]
         }
         self.scheme_name = kwargs.pop('scheme')
         self.titles = kwargs.pop('titles')
@@ -207,6 +218,7 @@ class NamePartsFormField(forms.MultiValueField):
             if fname == 'title' and self.scheme_titles:
                 d = dict(defaults)
                 d.pop('max_length', None)
+                d.pop('validators', None)
                 field = forms.ChoiceField(
                     **d,
                     choices=[('', '')] + [(d, d) for d in self.scheme_titles[1]]
@@ -215,6 +227,7 @@ class NamePartsFormField(forms.MultiValueField):
             elif fname == 'salutation':
                 d = dict(defaults)
                 d.pop('max_length', None)
+                d.pop('validators', None)
                 field = forms.ChoiceField(
                     **d,
                     choices=[('', '---')] + PERSON_NAME_SALUTATIONS
@@ -251,17 +264,14 @@ class WrappedPhonePrefixSelect(Select):
 
     def __init__(self, initial=None):
         choices = [("", "---------")]
-        language = get_babel_locale()  # changed from default implementation that used the django locale
-        locale = Locale(translation.to_locale(language))
-        for prefix, values in _COUNTRY_CODE_TO_REGION_CODE.items():
-            prefix = "+%d" % prefix
-            if initial and initial in values:
-                self.initial = prefix
-            for country_code in values:
-                country_name = locale.territories.get(country_code)
-                if country_name:
-                    choices.append((prefix, "{} {}".format(country_name, prefix)))
-        super().__init__(choices=sorted(choices, key=lambda item: item[1]), attrs={'aria-label': pgettext_lazy('phonenumber', 'International area code')})
+
+        if initial:
+            for prefix, values in _COUNTRY_CODE_TO_REGION_CODE.items():
+                if initial in values:
+                    self.initial = "+%d" % prefix
+                    break
+        choices += get_phone_prefixes_sorted_and_localized()
+        super().__init__(choices=choices, attrs={'aria-label': pgettext_lazy('phonenumber', 'International area code')})
 
     def render(self, name, value, *args, **kwargs):
         return super().render(name, value or self.initial, *args, **kwargs)
@@ -305,7 +315,12 @@ class WrappedPhoneNumberPrefixWidget(PhoneNumberPrefixWidget):
         silently deleting data.
         """
         if value:
-            if type(value) == PhoneNumber:
+            if isinstance(value, str):
+                try:
+                    value = PhoneNumber.from_string(value)
+                except:
+                    pass
+            if isinstance(value, PhoneNumber):
                 if value.country_code and value.national_number:
                     return [
                         "+%d" % value.country_code,
@@ -692,7 +707,7 @@ class BaseQuestionsForm(forms.Form):
                     label=label, required=required,
                     min_value=q.valid_number_min or Decimal('0.00'),
                     max_value=q.valid_number_max,
-                    help_text=q.help_text,
+                    help_text=help_text,
                     initial=initial.answer if initial else None,
                 )
             elif q.type == Question.TYPE_STRING:

src/pretix/base/forms/widgets.py

@@ -42,6 +42,24 @@ from django.utils.timezone import get_current_timezone, now
 from django.utils.translation import gettext_lazy as _
 
 
+def replace_arabic_numbers(inp):
+    if not isinstance(inp, str):
+        return inp
+    table = {
+        1632: 48,  # 0
+        1633: 49,  # 1
+        1634: 50,  # 2
+        1635: 51,  # 3
+        1636: 52,  # 4
+        1637: 53,  # 5
+        1638: 54,  # 6
+        1639: 55,  # 7
+        1640: 56,  # 8
+        1641: 57,  # 9
+    }
+    return inp.translate(table)
+
+
 class DatePickerWidget(forms.DateInput):
     def __init__(self, attrs=None, date_format=None):
         attrs = attrs or {}
@@ -62,6 +80,10 @@ class DatePickerWidget(forms.DateInput):
 
         forms.DateInput.__init__(self, date_attrs, date_format)
 
+    def value_from_datadict(self, data, files, name):
+        v = super().value_from_datadict(data, files, name)
+        return replace_arabic_numbers(v)
+
 
 class TimePickerWidget(forms.TimeInput):
     def __init__(self, attrs=None, time_format=None):
@@ -83,6 +105,10 @@ class TimePickerWidget(forms.TimeInput):
 
         forms.TimeInput.__init__(self, time_attrs, time_format)
 
+    def value_from_datadict(self, data, files, name):
+        v = super().value_from_datadict(data, files, name)
+        return replace_arabic_numbers(v)
+
 
 class UploadedFileWidget(forms.ClearableFileInput):
     def __init__(self, *args, **kwargs):
@@ -179,6 +205,10 @@ class SplitDateTimePickerWidget(forms.SplitDateTimeWidget):
         # Skip one hierarchy level
         forms.MultiWidget.__init__(self, widgets, attrs)
 
+    def value_from_datadict(self, data, files, name):
+        v = super().value_from_datadict(data, files, name)
+        return [replace_arabic_numbers(i) for i in v]
+
 
 class BusinessBooleanRadio(forms.RadioSelect):
     def __init__(self, require_business=False, attrs=None):

src/pretix/base/management/commands/check_order_transactions.py

@@ -104,4 +104,4 @@ class Command(BaseCommand):
             print(f"Error in order {o.full_code}: status={o.status}, sum(positions)+sum(fees)={o.position_total + o.fee_total}, "
                   f"order.total={o.total}, sum(transactions)={o.tx_total}, expected={o.correct_total}, pos_cnt={o.position_cnt}, tx_pos_cnt={o.tx_cnt}")
 
-        self.stderr.write(self.style.SUCCESS(f'Check completed.'))
+        self.stderr.write(self.style.SUCCESS('Check completed.'))

src/pretix/base/management/commands/shell_scoped.py

@@ -19,11 +19,13 @@
 # You should have received a copy of the GNU Affero General Public License along with this program.  If not, see
 # <https://www.gnu.org/licenses/>.
 #
+import logging
 import sys
 
 from django.apps import apps
 from django.core.management import call_command
 from django.core.management.base import BaseCommand
+from django.db import connection
 from django_scopes import scope, scopes_disabled
 
 
@@ -33,6 +35,13 @@ class Command(BaseCommand):
         parser.parse_args = lambda x: parser.parse_known_args(x)[0]
         return parser
 
+    def add_arguments(self, parser):
+        parser.add_argument(
+            '--print-sql',
+            action='store_true',
+            help='Print all SQL queries.',
+        )
+
     def handle(self, *args, **options):
         try:
             from django_extensions.management.commands import shell_plus  # noqa
@@ -41,6 +50,11 @@ class Command(BaseCommand):
             cmd = 'shell'
             del options['skip_checks']
 
+        if options['print_sql']:
+            connection.force_debug_cursor = True
+            logger = logging.getLogger("django.db.backends")
+            logger.setLevel(logging.DEBUG)
+
         parser = self.create_parser(sys.argv[0], sys.argv[1])
         flags = parser.parse_known_args(sys.argv[2:])[1]
         if "--override" in flags:

src/pretix/base/migrations/0208_auto_20220214_1632.py

@@ -0,0 +1,22 @@
+# Generated by Django 3.2.4 on 2022-02-14 16:32
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('pretixbase', '0207_auto_20220119_1427'),
+    ]
+
+    operations = [
+        migrations.AddField(
+            model_name='user',
+            name='auth_backend_identifier',
+            field=models.CharField(db_index=True, max_length=190, null=True),
+        ),
+        migrations.AlterUniqueTogether(
+            name='user',
+            unique_together={('auth_backend', 'auth_backend_identifier')},
+        ),
+    ]

src/pretix/base/migrations/0209_device_info.py

@@ -0,0 +1,18 @@
+# Generated by Django 3.2.12 on 2022-03-22 11:57
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('pretixbase', '0208_auto_20220214_1632'),
+    ]
+
+    operations = [
+        migrations.AddField(
+            model_name='device',
+            name='info',
+            field=models.JSONField(null=True),
+        ),
+    ]

src/pretix/base/models/auth.py

@@ -44,7 +44,7 @@ from django.contrib.auth.models import (
 )
 from django.contrib.auth.tokens import default_token_generator
 from django.contrib.contenttypes.models import ContentType
-from django.db import models
+from django.db import IntegrityError, models, transaction
 from django.db.models import Q
 from django.utils.crypto import get_random_string, salted_hmac
 from django.utils.timezone import now
@@ -61,6 +61,10 @@ from pretix.helpers.urls import build_absolute_uri
 from .base import LoggingMixin
 
 
+class EmailAddressTakenError(IntegrityError):
+    pass
+
+
 class UserManager(BaseUserManager):
     """
     This is the user manager for our custom user model. See the User
@@ -83,6 +87,116 @@ class UserManager(BaseUserManager):
         user.save()
         return user
 
+    def get_or_create_for_backend(self, backend, identifier, email, set_always, set_on_creation):
+        """
+        This method should be used by third-party authentication backends to log in a user.
+        It either returns an already existing user or creates a new user.
+
+        In pretix 4.7 and earlier, email addresses were the only property to identify a user with.
+        Starting with pretix 4.8, backends SHOULD instead use a unique, immutable identifier
+        based on their backend data store to allow for changing email addresses.
+
+        This method transparently handles the conversion of old user accounts and adds the
+        backend identifier to their database record.
+
+        This method will never return users managed by a different authentication backend.
+        If you try to create an account with an email address already blocked by a different
+        authentication backend, :py:class:`EmailAddressTakenError` will be raised. In this case,
+        you should display a message to the user.
+
+        :param backend: The `identifier` attribute of the authentication backend
+        :param identifier: The unique, immutable identifier of this user, max. 190 characters
+        :param email: The user's email address
+        :param set_always: A dictionary of fields to update on the user model on every login
+        :param set_on_creation: A dictionary of fields to set on the user model if it's newly created
+        :return: A `User` instance.
+        """
+        if identifier is None:
+            raise ValueError('You need to supply a custom, unique identifier for this user.')
+        if email is None:
+            raise ValueError('You need to supply an email address for this user.')
+        if 'auth_backend_identifier' in set_always or 'auth_backend_identifier' in set_on_creation or \
+                'auth_backend' in set_always or 'auth_backend' in set_on_creation:
+            raise ValueError('You may not update auth_backend/auth_backend_identifier.')
+        if len(identifier) > 190:
+            raise ValueError('The user identifier must not be more than 190 characters.')
+
+        # Always update the email address
+        set_always.update({'email': email})
+
+        # First, check if we find the user based on it's backend-specific authenticator
+        try:
+            u = self.get(
+                auth_backend=backend,
+                auth_backend_identifier=identifier,
+            )
+            dirty = False
+            for k, v in set_always.items():
+                if getattr(u, k) != v:
+                    setattr(u, k, v)
+                    dirty = True
+            if dirty:
+                try:
+                    with transaction.atomic():
+                        u.save(update_fields=set_always.keys())
+                except IntegrityError:
+                    # This might only raise IntegrityError if the email address is used
+                    # by someone else
+                    raise EmailAddressTakenError()
+            return u
+        except self.model.DoesNotExist:
+            pass
+
+        # Second, check if we find the user based on their email address and this backend
+        try:
+            u = self.get(
+                auth_backend=backend,
+                auth_backend_identifier__isnull=True,
+                email=email,
+            )
+            u.auth_backend_identifier = identifier
+            for k, v in set_always.items():
+                setattr(u, k, v)
+            try:
+                with transaction.atomic():
+                    u.save(update_fields=['auth_backend_identifier'] + list(set_always.keys()))
+                return u
+            except IntegrityError:
+                # This might only raise IntegrityError if this code is being executed twice
+                # and runs into a race condition, this mechanism is taken from Django's
+                # get_or_create
+                try:
+                    return self.get(
+                        auth_backend=backend,
+                        auth_backend_identifier=identifier,
+                    )
+                except self.model.DoesNotExist:
+                    pass
+                raise
+        except self.model.DoesNotExist:
+            pass
+
+        # Third, create a new user
+        u = User(
+            auth_backend=backend,
+            auth_backend_identifier=identifier,
+            **set_on_creation,
+            **set_always,
+        )
+        try:
+            u.save(force_insert=True)
+            return u
+        except IntegrityError:
+            # This might either be a race condition or the email address is taken
+            # by a different backend
+            try:
+                return self.get(
+                    auth_backend=backend,
+                    auth_backend_identifier=identifier,
+                )
+            except self.model.DoesNotExist:
+                raise EmailAddressTakenError()
+
 
 def generate_notifications_token():
     return get_random_string(length=32)
@@ -117,6 +231,10 @@ class User(AbstractBaseUser, PermissionsMixin, LoggingMixin):
     :type needs_password_change: bool
     :param timezone: The user's preferred timezone.
     :type timezone: str
+    :param auth_backend: The identifier of the authentication backend plugin responsible for managing this user.
+    :type auth_backend: str
+    :param auth_backend_identifier: The native identifier of the user provided by a non-native authentication backend.
+    :type auth_backend_identifier: str
     """
 
     USERNAME_FIELD = 'email'
@@ -152,6 +270,7 @@ class User(AbstractBaseUser, PermissionsMixin, LoggingMixin):
     )
     notifications_token = models.CharField(max_length=255, default=generate_notifications_token)
     auth_backend = models.CharField(max_length=255, default='native')
+    auth_backend_identifier = models.CharField(max_length=190, db_index=True, null=True, blank=True)
     session_token = models.CharField(max_length=32, default=generate_session_token)
 
     objects = UserManager()
@@ -164,6 +283,7 @@ class User(AbstractBaseUser, PermissionsMixin, LoggingMixin):
         verbose_name = _("User")
         verbose_name_plural = _("Users")
         ordering = ('email',)
+        unique_together = (('auth_backend', 'auth_backend_identifier'),)
 
     def save(self, *args, **kwargs):
         self.email = self.email.lower()
@@ -378,6 +498,23 @@ class User(AbstractBaseUser, PermissionsMixin, LoggingMixin):
             | Q(id__in=self.teams.filter(**kwargs).values_list('limit_events__id', flat=True))
         )
 
+    @scopes_disabled()
+    def get_organizers_with_any_permission(self, request=None):
+        """
+        Returns a queryset of organizers the user has any permissions to.
+
+        :param request: The current request (optional). Required to detect staff sessions properly.
+        :return: Iterable of Organizers
+        """
+        from .event import Organizer
+
+        if request and self.has_active_staff_session(request.session.session_key):
+            return Organizer.objects.all()
+
+        return Organizer.objects.filter(
+            id__in=self.teams.values_list('organizer', flat=True)
+        )
+
     @scopes_disabled()
     def get_organizers_with_permission(self, permission, request=None):
         """

src/pretix/base/models/checkin.py

@@ -221,7 +221,7 @@ class CheckinList(LoggedModel):
             return rules
 
         if operator in ('or', 'and') and seen_nonbool:
-            raise ValidationError(f'You cannot use OR/AND logic on a level below a comparison operator.')
+            raise ValidationError('You cannot use OR/AND logic on a level below a comparison operator.')
 
         for v in values:
             cls.validate_rules(v, seen_nonbool=seen_nonbool or operator not in ('or', 'and'), depth=depth + 1)

src/pretix/base/models/customers.py

@@ -172,6 +172,7 @@ class Customer(LoggedModel):
         return salted_hmac(key_salt, payload).hexdigest()
 
     def get_email_context(self):
+        from pretix.base.email import get_name_parts_localized
         ctx = {
             'name': self.name,
             'organizer': self.organizer.name,
@@ -180,7 +181,13 @@ class Customer(LoggedModel):
         for f, l, w in name_scheme['fields']:
             if f == 'full_name':
                 continue
-            ctx['name_%s' % f] = self.name_parts.get(f, '')
+            ctx['name_%s' % f] = get_name_parts_localized(self.name_parts, f)
+
+        if "concatenation_for_salutation" in name_scheme:
+            ctx['name_for_salutation'] = name_scheme["concatenation_for_salutation"](self.name_parts)
+        else:
+            ctx['name_for_salutation'] = name_scheme["concatenation"](self.name_parts)
+
         return ctx
 
     @property

src/pretix/base/models/devices.py

@@ -156,6 +156,9 @@ class Device(LoggedModel):
         null=True,
         blank=False
     )
+    info = models.JSONField(
+        null=True, blank=True,
+    )
 
     objects = ScopedManager(organizer='organizer')
 

src/pretix/base/models/event.py

@@ -1179,21 +1179,21 @@ class Event(EventMixin, LoggedModel):
             if not p.name.startswith('.') and getattr(p, 'visible', True)
         }
 
-    def set_active_plugins(self, modules, allow_restricted=False):
+    def set_active_plugins(self, modules, allow_restricted=frozenset()):
         plugins_active = self.get_plugins()
         plugins_available = self.get_available_plugins()
 
         enable = [m for m in modules if m not in plugins_active and m in plugins_available]
 
         for module in enable:
-            if getattr(plugins_available[module].app, 'restricted', False) and not allow_restricted:
+            if getattr(plugins_available[module].app, 'restricted', False) and module not in allow_restricted:
                 modules.remove(module)
             elif hasattr(plugins_available[module].app, 'installed'):
                 getattr(plugins_available[module].app, 'installed')(self)
 
         self.plugins = ",".join(modules)
 
-    def enable_plugin(self, module, allow_restricted=False):
+    def enable_plugin(self, module, allow_restricted=frozenset()):
         plugins_active = self.get_plugins()
         from pretix.presale.style import regenerate_css
 

src/pretix/base/models/items.py

@@ -44,7 +44,7 @@ import dateutil.parser
 import pytz
 from django.conf import settings
 from django.core.exceptions import ValidationError
-from django.core.validators import RegexValidator
+from django.core.validators import MinValueValidator, RegexValidator
 from django.db import models
 from django.db.models import Q
 from django.utils import formats
@@ -479,12 +479,14 @@ class Item(LoggedModel):
     min_per_order = models.IntegerField(
         verbose_name=_('Minimum amount per order'),
         null=True, blank=True,
+        validators=[MinValueValidator(0)],
         help_text=_('This product can only be bought if it is added to the cart at least this many times. If you keep '
                     'the field empty or set it to 0, there is no special limit for this product.')
     )
     max_per_order = models.IntegerField(
         verbose_name=_('Maximum amount per order'),
         null=True, blank=True,
+        validators=[MinValueValidator(0)],
         help_text=_('This product can only be bought at most this many times within one order. If you keep the field '
                     'empty or set it to 0, there is no special limit for this product. The limit for the maximum '
                     'number of items in the whole order applies regardless.')
@@ -1697,7 +1699,7 @@ class Quota(LoggedModel):
             if event != item.event:
                 raise ValidationError(_('One or more items do not belong to this event.'))
             if item.has_variations:
-                if not any(var.item == item for var in variations):
+                if not variations or not any(var.item == item for var in variations):
                     raise ValidationError(_('One or more items has variations but none of these are in the variations list.'))
 
     @staticmethod

src/pretix/base/models/orders.py

@@ -638,12 +638,13 @@ class Order(LockModel, LoggedModel):
                 return False
         if self.user_cancel_deadline and now() > self.user_cancel_deadline:
             return False
-        if self.status == Order.STATUS_PENDING:
-            return self.event.settings.cancel_allow_user
-        elif self.status == Order.STATUS_PAID:
+
+        if self.status == Order.STATUS_PAID or self.payment_refund_sum > Decimal('0.00'):
             if self.total == Decimal('0.00'):
                 return self.event.settings.cancel_allow_user
             return self.event.settings.cancel_allow_user_paid
+        elif self.status == Order.STATUS_PENDING:
+            return self.event.settings.cancel_allow_user
         return False
 
     def propose_auto_refunds(self, amount: Decimal, payments: list=None):
@@ -976,7 +977,7 @@ class Order(LockModel, LoggedModel):
             SendMailException, TolerantDict, mail, render_mail,
         )
 
-        if not self.email:
+        if not self.email and not (position and position.attendee_email):
             return
 
         for k, v in self.event.meta_data.items():
@@ -1330,6 +1331,10 @@ class AbstractPosition(models.Model):
         else:
             return {}
 
+    @property
+    def item_and_variation(self):
+        return self.item, self.variation
+
     @meta_info_data.setter
     def meta_info_data(self, d):
         self.meta_info = json.dumps(d)
@@ -1724,10 +1729,10 @@ class OrderPayment(models.Model):
             email_context = get_email_context(event=self.order.event, order=self.order, position=position)
             email_subject = _('Event registration confirmed: %(code)s') % {'code': self.order.code}
             try:
-                self.order.send_mail(
+                position.send_mail(
                     email_subject, email_template, email_context,
                     'pretix.event.order.email.order_paid', user,
-                    invoices=[], position=position,
+                    invoices=[],
                     attach_tickets=True,
                     attach_ical=self.order.event.settings.mail_attach_ical
                 )

src/pretix/base/models/seating.py

@@ -26,7 +26,7 @@ import jsonschema
 from django.contrib.staticfiles import finders
 from django.core.exceptions import ValidationError
 from django.db import models
-from django.db.models import Exists, F, OuterRef, Q, Value
+from django.db.models import Exists, F, OuterRef, Q, Subquery, Value
 from django.db.models.functions import Power
 from django.utils.deconstruct import deconstructible
 from django.utils.timezone import now
@@ -281,10 +281,26 @@ class Seat(models.Model):
             q = Q(has_order=True) | Q(has_voucher=True)
             if ignore_cart is not True:
                 q |= Q(has_cart=True)
+
+            # The following looks like it makes no sense. Why wouldn't we just use ``Value(self.x)``, we already now
+            # the value? The reason is that x and y are floating point values generated from our JSON files. As it turns
+            # out, PostgreSQL MIGHT store floating point values with a different precision based on the underlying system
+            # architecture. So if we generate e.g. 670.247128887222289 from the JSON file and store it to the database,
+            # PostgreSQL will store it as 670.247128887222289 internally. However if we query it again, we only get
+            # 670.247128887222 back. But if we do calculations with a field in PostgreSQL itself, it uses the full
+            # precision for the calculation.
+            # We don't actually care about the results with this precision, but we care that the results from this
+            # function are exactly the same as from event.free_seats(), so we do this subquery trick to deal with
+            # PostgreSQL's internal values in both cases.
+            # In the long run, we probably just want to round the numbers on insert...
+            # See also https://www.postgresql.org/docs/11/runtime-config-client.html#GUC-EXTRA-FLOAT-DIGITS
+            self_x = Subquery(Seat.objects.filter(pk=self.pk).values('x'))
+            self_y = Subquery(Seat.objects.filter(pk=self.pk).values('y'))
+
             qs_closeby_taken = qs_annotated.annotate(
                 distance=(
-                    Power(F('x') - Value(self.x), Value(2), output_field=models.FloatField()) +
-                    Power(F('y') - Value(self.y), Value(2), output_field=models.FloatField())
+                    Power(F('x') - self_x, Value(2), output_field=models.FloatField()) +
+                    Power(F('y') - self_y, Value(2), output_field=models.FloatField())
                 )
             ).exclude(pk=self.pk).filter(
                 q,

src/pretix/base/payment.py

@@ -955,6 +955,8 @@ class BoxOfficeProvider(BasePaymentProvider):
         return {
             "pos_id": payment.info_data.get('pos_id', None),
             "receipt_id": payment.info_data.get('receipt_id', None),
+            "payment_type": payment.info_data.get('payment_type', None),
+            "payment_data": payment.info_data.get('payment_data', {}),
         }
 
     def payment_control_render(self, request, payment) -> str:

src/pretix/base/pdf.py

@@ -37,6 +37,7 @@ import hashlib
 import itertools
 import logging
 import os
+import re
 import subprocess
 import tempfile
 import uuid
@@ -48,12 +49,14 @@ from arabic_reshaper import ArabicReshaper
 from bidi.algorithm import get_display
 from django.conf import settings
 from django.contrib.staticfiles import finders
+from django.db.models import Max, Min
 from django.dispatch import receiver
 from django.utils.formats import date_format
 from django.utils.functional import SimpleLazyObject
 from django.utils.html import conditional_escape
 from django.utils.timezone import now
 from django.utils.translation import gettext_lazy as _
+from i18nfield.strings import LazyI18nString
 from PyPDF2 import PdfFileReader
 from pytz import timezone
 from reportlab.graphics import renderPDF
@@ -202,6 +205,11 @@ DEFAULT_VARIABLES = OrderedDict((
         "editor_sample": 'foo@bar.com',
         "evaluate": lambda op, order, ev: op.attendee_email or (op.addon_to.attendee_email if op.addon_to else '')
     }),
+    ("pseudonymization_id", {
+        "label": _("Pseudonymization ID (lead scanning)"),
+        "editor_sample": "GG89JUJDTA",
+        "evaluate": lambda orderposition, order, event: orderposition.pseudonymization_id,
+    }),
     ("event_name", {
         "label": _("Event name"),
         "editor_sample": _("Sample event name"),
@@ -387,30 +395,41 @@ DEFAULT_VARIABLES = OrderedDict((
     ("seat", {
         "label": _("Seat: Full name"),
         "editor_sample": _("Ground floor, Row 3, Seat 4"),
-        "evaluate": lambda op, order, ev: str(op.seat if op.seat else
+        "evaluate": lambda op, order, ev: str(get_seat(op) if get_seat(op) else
                                               _('General admission') if ev.seating_plan_id is not None else "")
     }),
     ("seat_zone", {
         "label": _("Seat: zone"),
         "editor_sample": _("Ground floor"),
-        "evaluate": lambda op, order, ev: str(op.seat.zone_name if op.seat else
+        "evaluate": lambda op, order, ev: str(get_seat(op).zone_name if get_seat(op) else
                                               _('General admission') if ev.seating_plan_id is not None else "")
     }),
     ("seat_row", {
         "label": _("Seat: row"),
         "editor_sample": "3",
-        "evaluate": lambda op, order, ev: str(op.seat.row_name if op.seat else "")
+        "evaluate": lambda op, order, ev: str(get_seat(op).row_name if get_seat(op) else "")
     }),
     ("seat_number", {
         "label": _("Seat: seat number"),
         "editor_sample": 4,
-        "evaluate": lambda op, order, ev: str(op.seat.seat_number if op.seat else "")
+        "evaluate": lambda op, order, ev: str(get_seat(op).seat_number if get_seat(op) else "")
     }),
     ("first_scan", {
         "label": _("Date and time of first scan"),
         "editor_sample": _("2017-05-31 19:00"),
         "evaluate": lambda op, order, ev: get_first_scan(op)
     }),
+    ("giftcard_issuance_date", {
+
+        "label": _("Gift card: Issuance date"),
+        "editor_sample": _("2017-05-31"),
+        "evaluate": lambda op, order, ev: get_giftcard_issuance(op, ev)
+    }),
+    ("giftcard_expiry_date", {
+        "label": _("Gift card: Expiration date"),
+        "editor_sample": _("2017-05-31"),
+        "evaluate": lambda op, order, ev: get_giftcard_expiry(op, ev)
+    }),
 ))
 DEFAULT_IMAGES = OrderedDict([])
 
@@ -485,10 +504,17 @@ def variables_from_questions(sender, *args, **kwargs):
     for q in sender.questions.all():
         if q.type == Question.TYPE_FILE:
             continue
+        d['question_{}'.format(q.identifier)] = {
+            'label': _('Question: {question}').format(question=q.question),
+            'editor_sample': _('<Answer: {question}>').format(question=q.question),
+            'evaluate': partial(get_answer, question_id=q.pk),
+            'migrate_from': 'question_{}'.format(q.pk)
+        }
         d['question_{}'.format(q.pk)] = {
             'label': _('Question: {question}').format(question=q.question),
             'editor_sample': _('<Answer: {question}>').format(question=q.question),
-            'evaluate': partial(get_answer, question_id=q.pk)
+            'evaluate': partial(get_answer, question_id=q.pk),
+            'hidden': True,
         }
     return d
 
@@ -546,6 +572,24 @@ def get_variables(event):
     return v
 
 
+def get_giftcard_expiry(op: OrderPosition, ev):
+    if not op.item.issue_giftcard:
+        return ""  # performance optimization
+    m = op.issued_gift_cards.aggregate(m=Min('expires'))['m']
+    if not m:
+        return ""
+    return date_format(m.astimezone(ev.timezone), "SHORT_DATE_FORMAT")
+
+
+def get_giftcard_issuance(op: OrderPosition, ev):
+    if not op.item.issue_giftcard:
+        return ""  # performance optimization
+    m = op.issued_gift_cards.aggregate(m=Max('issuance'))['m']
+    if not m:
+        return ""
+    return date_format(m.astimezone(ev.timezone), "SHORT_DATE_FORMAT")
+
+
 def get_first_scan(op: OrderPosition):
     scans = list(op.checkins.all())
 
@@ -557,6 +601,14 @@ def get_first_scan(op: OrderPosition):
     return ""
 
 
+def get_seat(op: OrderPosition):
+    if op.seat_id:
+        return op.seat
+    if op.addon_to_id:
+        return op.addon_to.seat
+    return None
+
+
 reshaper = SimpleLazyObject(lambda: ArabicReshaper(configuration={
     'delete_harakat': True,
     'support_ligatures': False,
@@ -616,12 +668,14 @@ class Renderer:
                          preserveAspectRatio=True, anchor='n',
                          mask='auto')
 
-    def _draw_barcodearea(self, canvas: Canvas, op: OrderPosition, o: dict):
+    def _draw_barcodearea(self, canvas: Canvas, op: OrderPosition, order: Order, o: dict):
         content = o.get('content', 'secret')
         if content == 'secret':
+            # do not use get_text_content because it uses a shortened version of secret
+            # and does not deal with our default value here properly
             content = op.secret
-        elif content == 'pseudonymization_id':
-            content = op.pseudonymization_id
+        else:
+            content = self._get_text_content(op, order, o)
 
         level = 'H'
         if len(content) > 32:
@@ -648,20 +702,51 @@ class Renderer:
                 return self._get_text_content(op, order, o, True)
 
         ev = self._get_ev(op, order)
+
         if not o['content']:
             return '(error)'
-        if o['content'] == 'other':
-            return o['text']
+
+        if o['content'] == 'other' or o['content'] == 'other_i18n':
+            if o['content'] == 'other_i18n':
+                text = str(LazyI18nString(o['text_i18n']))
+            else:
+                text = o['text']
+
+            def replace(x):
+                print(x.group(1))
+                if x.group(1).startswith('itemmeta:'):
+                    return op.item.meta_data.get(x.group(1)[9:]) or ''
+                elif x.group(1).startswith('meta:'):
+                    return ev.meta_data.get(x.group(1)[5:]) or ''
+                elif x.group(1) not in self.variables:
+                    return x.group(0)
+                if x.group(1) == 'secret':
+                    # Do not use shortened version
+                    return op.secret
+
+                try:
+                    return self.variables[x.group(1)]['evaluate'](op, order, ev)
+                except:
+                    logger.exception('Failed to process variable.')
+                    return '(error)'
+
+            # We do not use str.format like in emails so we (a) can evaluate lazily and (b) can re-implement this
+            # 1:1 on other platforms that render PDFs through our API (libpretixprint)
+            return re.sub(r'\{([a-zA-Z0-9:_]+)\}', replace, text)
+
         elif o['content'].startswith('itemmeta:'):
             return op.item.meta_data.get(o['content'][9:]) or ''
+
         elif o['content'].startswith('meta:'):
             return ev.meta_data.get(o['content'][5:]) or ''
+
         elif o['content'] in self.variables:
             try:
                 return self.variables[o['content']]['evaluate'](op, order, ev)
             except:
                 logger.exception('Failed to process variable.')
                 return '(error)'
+
         return ''
 
     def _draw_imagearea(self, canvas: Canvas, op: OrderPosition, order: Order, o: dict):
@@ -754,20 +839,30 @@ class Renderer:
             p.drawOn(canvas, 0, -h - ad[1])
         canvas.restoreState()
 
-    def draw_page(self, canvas: Canvas, order: Order, op: OrderPosition, show_page=True):
-        for o in self.layout:
-            if o['type'] == "barcodearea":
-                self._draw_barcodearea(canvas, op, o)
-            elif o['type'] == "imagearea":
-                self._draw_imagearea(canvas, op, order, o)
-            elif o['type'] == "textarea":
-                self._draw_textarea(canvas, op, order, o)
-            elif o['type'] == "poweredby":
-                self._draw_poweredby(canvas, op, o)
-            if self.bg_pdf:
-                canvas.setPageSize((self.bg_pdf.getPage(0).mediaBox[2], self.bg_pdf.getPage(0).mediaBox[3]))
-        if show_page:
-            canvas.showPage()
+    def draw_page(self, canvas: Canvas, order: Order, op: OrderPosition, show_page=True, only_page=None):
+        page_count = self.bg_pdf.getNumPages()
+
+        if not only_page and not show_page:
+            raise ValueError("only_page=None and show_page=False cannot be combined")
+
+        for page in range(page_count):
+            if only_page and only_page != page + 1:
+                continue
+            for o in self.layout:
+                if o.get('page', 1) != page + 1:
+                    continue
+                if o['type'] == "barcodearea":
+                    self._draw_barcodearea(canvas, op, order, o)
+                elif o['type'] == "imagearea":
+                    self._draw_imagearea(canvas, op, order, o)
+                elif o['type'] == "textarea":
+                    self._draw_textarea(canvas, op, order, o)
+                elif o['type'] == "poweredby":
+                    self._draw_poweredby(canvas, op, o)
+                if self.bg_pdf:
+                    canvas.setPageSize((self.bg_pdf.getPage(page).mediaBox[2], self.bg_pdf.getPage(page).mediaBox[3]))
+            if show_page:
+                canvas.showPage()
 
     def render_background(self, buffer, title=_('Ticket')):
         if settings.PDFTK:
@@ -780,7 +875,7 @@ class Renderer:
                 subprocess.run([
                     settings.PDFTK,
                     os.path.join(d, 'front.pdf'),
-                    'background',
+                    'multibackground',
                     os.path.join(d, 'back.pdf'),
                     'output',
                     os.path.join(d, 'out.pdf'),
@@ -794,8 +889,8 @@ class Renderer:
             new_pdf = PdfFileReader(buffer)
             output = PdfFileWriter()
 
-            for page in new_pdf.pages:
-                bg_page = copy.copy(self.bg_pdf.getPage(0))
+            for i, page in enumerate(new_pdf.pages):
+                bg_page = copy.copy(self.bg_pdf.getPage(i))
                 bg_page.mergePage(page)
                 output.addPage(bg_page)
 

src/pretix/base/secrets.py

@@ -247,7 +247,7 @@ def assign_ticket_secret(event, position, force_invalidate_if_revokation_list_us
         **kwargs
     )
     changed = position.secret != secret
-    if position.secret and changed and gen.use_revocation_list:
+    if position.secret and changed and gen.use_revocation_list and position.pk:
         position.revoked_secrets.create(event=event, secret=position.secret)
     position.secret = secret
     if save and changed:

src/pretix/base/services/cancelevent.py

@@ -214,8 +214,8 @@ def cancel_event(self, event: Event, subevent: int, auto_refund: bool,
             refund_amount = o.payment_refund_sum
 
             try:
-                if auto_refund:
-                    _try_auto_refund(o.pk, manual_refund=manual_refund, allow_partial=True,
+                if auto_refund or manual_refund:
+                    _try_auto_refund(o.pk, auto_refund=auto_refund, manual_refund=manual_refund, allow_partial=True,
                                      source=OrderRefund.REFUND_SOURCE_ADMIN, refund_as_giftcard=refund_as_giftcard,
                                      giftcard_expires=giftcard_expires, giftcard_conditions=giftcard_conditions,
                                      comment=gettext('Event canceled'))
@@ -272,8 +272,8 @@ def cancel_event(self, event: Event, subevent: int, auto_refund: bool,
             ocm.commit()
             refund_amount = o.payment_refund_sum - o.total
 
-            if auto_refund:
-                _try_auto_refund(o.pk, manual_refund=manual_refund, allow_partial=True,
+            if auto_refund or manual_refund:
+                _try_auto_refund(o.pk, auto_refund=auto_refund, manual_refund=manual_refund, allow_partial=True,
                                  source=OrderRefund.REFUND_SOURCE_ADMIN, refund_as_giftcard=refund_as_giftcard,
                                  giftcard_expires=giftcard_expires, giftcard_conditions=giftcard_conditions,
                                  comment=gettext('Event canceled'))

src/pretix/base/services/checkin.py

@@ -373,22 +373,22 @@ class SQLLogic:
                 ).astimezone(pytz.UTC))
             elif values[0] == 'date_from':
                 return Coalesce(
-                    F(f'subevent__date_from'),
-                    F(f'order__event__date_from'),
+                    F('subevent__date_from'),
+                    F('order__event__date_from'),
                 )
             elif values[0] == 'date_to':
                 return Coalesce(
-                    F(f'subevent__date_to'),
-                    F(f'subevent__date_from'),
-                    F(f'order__event__date_to'),
-                    F(f'order__event__date_from'),
+                    F('subevent__date_to'),
+                    F('subevent__date_from'),
+                    F('order__event__date_to'),
+                    F('order__event__date_from'),
                 )
             elif values[0] == 'date_admission':
                 return Coalesce(
-                    F(f'subevent__date_admission'),
-                    F(f'subevent__date_from'),
-                    F(f'order__event__date_admission'),
-                    F(f'order__event__date_from'),
+                    F('subevent__date_admission'),
+                    F('subevent__date_from'),
+                    F('order__event__date_admission'),
+                    F('order__event__date_from'),
                 )
             else:
                 raise ValueError(f'Unknown time type {values[0]}')

src/pretix/base/services/export.py

@@ -56,6 +56,8 @@ def export(self, event: Event, fileid: str, provider: str, form_data: Dict[str,
     with language(event.settings.locale, event.settings.region), override(event.settings.timezone):
         responses = register_data_exporters.send(event)
         for receiver, response in responses:
+            if not response:
+                continue
             ex = response(event, event.organizer, set_progress)
             if ex.identifier == provider:
                 d = ex.render(form_data)

src/pretix/base/services/mail.py

@@ -77,7 +77,7 @@ from pretix.base.signals import email_filter, global_email_filter
 from pretix.celery_app import app
 from pretix.helpers.hierarkey import clean_filename
 from pretix.multidomain.urlreverse import build_absolute_uri
-from pretix.presale.ical import get_ical
+from pretix.presale.ical import get_private_icals
 
 logger = logging.getLogger('pretix.base.mail')
 INVALID_ADDRESS = 'invalid-pretix-mail-address'
@@ -217,7 +217,7 @@ def mail(email: Union[str, Sequence[str]], subject: str, template: Union[str, La
                 for bcc_mail in settings_holder.settings.mail_bcc.split(','):
                     bcc.append(bcc_mail.strip())
 
-            if settings_holder.settings.mail_from not in (settings.DEFAULT_FROM_EMAIL, settings.MAIL_FROM_ORGANIZERS) \
+            if settings_holder.settings.mail_from in (settings.DEFAULT_FROM_EMAIL, settings.MAIL_FROM_ORGANIZERS) \
                     and settings_holder.settings.contact_mail and not headers.get('Reply-To'):
                 headers['Reply-To'] = settings_holder.settings.contact_mail
 
@@ -430,18 +430,7 @@ def mail_send_task(self, *args, to: List[str], subject: str, body: str, html: st
                                     }
                                 )
                         if attach_ical:
-                            ical_events = set()
-                            if event.has_subevents:
-                                if position:
-                                    ical_events.add(position.subevent)
-                                else:
-                                    for p in order.positions.all():
-                                        ical_events.add(p.subevent)
-                            else:
-                                ical_events.add(order.event)
-
-                            for i, e in enumerate(ical_events):
-                                cal = get_ical([e])
+                            for i, cal in enumerate(get_private_icals(event, [position] if position else order.positions.all())):
                                 email.attach('event-{}.ics'.format(i), cal.serialize(), 'text/calendar')
 
             email = email_filter.send_chained(event, 'message', message=email, order=order, user=user)
@@ -496,14 +485,17 @@ def mail_send_task(self, *args, to: List[str], subject: str, body: str, html: st
         try:
             backend.send_messages([email])
         except (smtplib.SMTPResponseException, smtplib.SMTPSenderRefused) as e:
-            if e.smtp_code in (101, 111, 421, 422, 431, 432, 442, 447, 452):
-                if e.smtp_code == 432 and settings.HAS_REDIS:
+            if e.smtp_code in (101, 111, 421, 422, 431, 432, 442, 447, 452, 454):
+                if e.smtp_code in (432, 454) and settings.HAS_REDIS:
                     # This is likely Microsoft Exchange Online which has a pretty bad rate limit of max. 3 concurrent
                     # SMTP connections which is *easily* exceeded with many celery threads. Just retrying with exponential
                     # backoff won't be good enough if we have a lot of emails, instead we'll need to make sure our retry
                     # intervals scatter such that the email won't all be retried at the same time again and cause the
                     # same problem.
                     # See also https://docs.microsoft.com/en-us/exchange/troubleshoot/send-emails/smtp-submission-improvements
+                    #
+                    # Also observed on AWS SES with 454:
+                    # https://aws.amazon.com/blogs/messaging-and-targeting/how-to-handle-a-throttling-maximum-sending-rate-exceeded-error/
                     from django_redis import get_redis_connection
 
                     redis_key = "pretix_mail_retry_" + hashlib.sha1(f"{getattr(backend, 'username', '_')}@{getattr(backend, 'host', '_')}".encode()).hexdigest()
@@ -590,7 +582,7 @@ def mail_send_task(self, *args, to: List[str], subject: str, body: str, html: st
                             }
                         )
                     raise e
-            if logger:
+            if log_target:
                 log_target.log_action(
                     'pretix.email.error',
                     data={

src/pretix/base/services/orders.py

@@ -53,7 +53,7 @@ from django.db.transaction import get_connection
 from django.dispatch import receiver
 from django.utils.functional import cached_property
 from django.utils.timezone import make_aware, now
-from django.utils.translation import gettext as _
+from django.utils.translation import gettext as _, gettext_lazy
 from django_scopes import scopes_disabled
 
 from pretix.api.models import OAuthApplication
@@ -384,7 +384,7 @@ def deny_order(order, comment='', user=None, send_mail: bool=True, auth=None):
 
 
 def _cancel_order(order, user=None, send_mail: bool=True, api_token=None, device=None, oauth_application=None,
-                  cancellation_fee=None, keep_fees=None, cancel_invoice=True):
+                  cancellation_fee=None, keep_fees=None, cancel_invoice=True, comment=None):
     """
     Mark this order as canceled
     :param order: The order to change
@@ -481,7 +481,7 @@ def _cancel_order(order, user=None, send_mail: bool=True, api_token=None, device
                     Voucher.objects.filter(pk=position.voucher.pk).update(redeemed=Greatest(0, F('redeemed') - 1))
 
         order.log_action('pretix.event.order.canceled', user=user, auth=api_token or oauth_application or device,
-                         data={'cancellation_fee': cancellation_fee})
+                         data={'cancellation_fee': cancellation_fee, 'comment': comment})
         order.cancellation_requests.all().delete()
 
         order.create_transactions()
@@ -489,7 +489,7 @@ def _cancel_order(order, user=None, send_mail: bool=True, api_token=None, device
         if send_mail:
             email_template = order.event.settings.mail_text_order_canceled
             with language(order.locale, order.event.settings.region):
-                email_context = get_email_context(event=order.event, order=order)
+                email_context = get_email_context(event=order.event, order=order, comment=comment or "")
                 email_subject = _('Order canceled: %(code)s') % {'code': order.code}
                 try:
                     order.send_mail(
@@ -932,16 +932,16 @@ def _create_order(event: Event, email: str, positions: List[CartPosition], now_d
 
 
 def _order_placed_email(event: Event, order: Order, pprov: BasePaymentProvider, email_template, log_entry: str,
-                        invoice, payment: OrderPayment):
+                        invoice, payment: OrderPayment, is_free=False):
     email_context = get_email_context(event=event, order=order, payment=payment if pprov else None)
-    email_subject = _('Your order: %(code)s') % {'code': order.code}
+    email_subject = gettext_lazy('Your order: {code}')
     try:
         order.send_mail(
             email_subject, email_template, email_context,
             log_entry,
             invoices=[invoice] if invoice and event.settings.invoice_email_attachment else [],
             attach_tickets=True,
-            attach_ical=event.settings.mail_attach_ical,
+            attach_ical=event.settings.mail_attach_ical and (not event.settings.mail_attach_ical_paid_only or is_free),
             attach_other_files=[a for a in [
                 event.settings.get('mail_attachment_new_order', as_type=str, default='')[len('file://'):]
             ] if a],
@@ -950,18 +950,17 @@ def _order_placed_email(event: Event, order: Order, pprov: BasePaymentProvider,
         logger.exception('Order received email could not be sent')
 
 
-def _order_placed_email_attendee(event: Event, order: Order, position: OrderPosition, email_template, log_entry: str):
+def _order_placed_email_attendee(event: Event, order: Order, position: OrderPosition, email_template, log_entry: str, is_free=False):
     email_context = get_email_context(event=event, order=order, position=position)
-    email_subject = _('Your event registration: %(code)s') % {'code': order.code}
+    email_subject = gettext_lazy('Your event registration: {code}')
 
     try:
-        order.send_mail(
+        position.send_mail(
             email_subject, email_template, email_context,
             log_entry,
             invoices=[],
             attach_tickets=True,
-            position=position,
-            attach_ical=event.settings.mail_attach_ical,
+            attach_ical=event.settings.mail_attach_ical and (not event.settings.mail_attach_ical_paid_only or is_free),
             attach_other_files=[a for a in [
                 event.settings.get('mail_attachment_new_order', as_type=str, default='')[len('file://'):]
             ] if a],
@@ -1070,11 +1069,13 @@ def _perform_order(event: Event, payment_provider: str, position_ids: List[str],
             email_attendees_template = event.settings.mail_text_order_placed_attendee
 
         if sales_channel in event.settings.mail_sales_channel_placed_paid:
-            _order_placed_email(event, order, pprov, email_template, log_entry, invoice, payment)
+            _order_placed_email(event, order, pprov, email_template, log_entry, invoice, payment,
+                                is_free=free_order_flow)
             if email_attendees:
                 for p in order.positions.all():
                     if p.addon_to_id is None and p.attendee_email and p.attendee_email != order.email:
-                        _order_placed_email_attendee(event, order, p, email_attendees_template, log_entry)
+                        _order_placed_email_attendee(event, order, p, email_attendees_template, log_entry,
+                                                     is_free=free_order_flow)
 
     return order.id
 
@@ -1466,7 +1467,7 @@ class OrderChangeManager:
         if self.order.event.settings.invoice_include_free or position.price != Decimal('0.00'):
             self._invoice_dirty = True
 
-    def add_position(self, item: Item, variation: ItemVariation, price: Decimal, addon_to: Order = None,
+    def add_position(self, item: Item, variation: ItemVariation, price: Decimal, addon_to: OrderPosition = None,
                      subevent: SubEvent = None, seat: Seat = None, membership: Membership = None):
         if isinstance(seat, str):
             if not seat:
@@ -1491,6 +1492,8 @@ class OrderChangeManager:
 
         if price is None:
             raise OrderError(self.error_messages['product_invalid'])
+        if item.variations.exists() and not variation:
+            raise OrderError(self.error_messages['product_without_variation'])
         if not addon_to and item.category and item.category.is_addon:
             raise OrderError(self.error_messages['addon_to_required'])
         if addon_to:
@@ -1526,6 +1529,8 @@ class OrderChangeManager:
             self._invoice_dirty = True
 
         self._operations.append(self.SplitOperation(position))
+        for a in position.addons.all():
+            self._operations.append(self.SplitOperation(a))
 
     def set_addons(self, addons):
         if self._operations:
@@ -1592,21 +1597,22 @@ class OrderChangeManager:
 
             op = opcache[a['addon_to']]
             item = _items_cache[a['item']]
+            subevent = op.subevent  # for now, we might lift this requirement later
             variation = _variations_cache[a['variation']] if a['variation'] is not None else None
 
             if item.category_id not in available_categories[op.pk]:
                 raise OrderError(error_messages['addon_invalid_base'])
 
             # Fetch all quotas. If there are no quotas, this item is not allowed to be sold.
-            quotas = list(item.quotas.filter(subevent=op.subevent)
-                          if variation is None else variation.quotas.filter(subevent=op.subevent))
+            quotas = list(item.quotas.filter(subevent=subevent)
+                          if variation is None else variation.quotas.filter(subevent=subevent))
             if not quotas:
                 raise OrderError(error_messages['unavailable'])
 
             if (a['item'], a['variation']) in input_addons[op.id]:
                 raise OrderError(error_messages['addon_duplicate_item'])
 
-            if item.require_voucher or op.item.hide_without_voucher or (op.variation and op.variation.hide_without_voucher):
+            if item.require_voucher or item.hide_without_voucher or (variation and variation.hide_without_voucher):
                 raise OrderError(error_messages['voucher_required'])
 
             if not item.is_available() or (variation and not variation.is_available()):
@@ -1616,11 +1622,11 @@ class OrderChangeManager:
                     variation and self.order.sales_channel not in variation.sales_channels):
                 raise OrderError(error_messages['unavailable'])
 
-            if op.subevent and item.pk in op.subevent.item_overrides and not op.subevent.item_overrides[op.item.pk].is_available():
+            if subevent and item.pk in subevent.item_overrides and not subevent.item_overrides[item.pk].is_available():
                 raise OrderError(error_messages['not_for_sale'])
 
-            if op.subevent and variation and variation.pk in op.subevent.var_overrides and \
-                    not op.subevent.var_overrides[variation.pk].is_available():
+            if subevent and variation and variation.pk in subevent.var_overrides and \
+                    not subevent.var_overrides[variation.pk].is_available():
                 raise OrderError(error_messages['not_for_sale'])
 
             if item.has_variations and not variation:
@@ -1629,10 +1635,10 @@ class OrderChangeManager:
             if variation and variation.item_id != item.pk:
                 raise OrderError(error_messages['not_for_sale'])
 
-            if op.subevent and op.subevent.presale_start and now() < op.subevent.presale_start:
+            if subevent and subevent.presale_start and now() < subevent.presale_start:
                 raise OrderError(error_messages['not_started'])
 
-            if (op.subevent and op.subevent.presale_has_ended) or self.event.presale_has_ended:
+            if (subevent and subevent.presale_has_ended) or self.event.presale_has_ended:
                 raise OrderError(error_messages['ended'])
 
             if item.require_bundling:
@@ -2381,7 +2387,8 @@ def perform_order(self, event: Event, payment_provider: str, positions: List[str
 _unset = object()
 
 
-def _try_auto_refund(order, manual_refund=False, allow_partial=False, source=OrderRefund.REFUND_SOURCE_BUYER,
+def _try_auto_refund(order, auto_refund=True, manual_refund=False, allow_partial=False,
+                     source=OrderRefund.REFUND_SOURCE_BUYER,
                      refund_as_giftcard=False, giftcard_expires=_unset, giftcard_conditions=None, comment=None):
     notify_admin = False
     error = False
@@ -2391,9 +2398,9 @@ def _try_auto_refund(order, manual_refund=False, allow_partial=False, source=Ord
     if refund_amount <= Decimal('0.00'):
         return
 
+    can_auto_refund_sum = 0
+
     if refund_as_giftcard:
-        proposals = {}
-        can_auto_refund = True
         can_auto_refund_sum = refund_amount
         with transaction.atomic():
             giftcard = order.event.organizer.issued_gift_cards.create(
@@ -2433,42 +2440,41 @@ def _try_auto_refund(order, manual_refund=False, allow_partial=False, source=Ord
                 if r.state != OrderRefund.REFUND_STATE_DONE:
                     notify_admin = True
 
-    else:
+    elif auto_refund:
         proposals = order.propose_auto_refunds(refund_amount)
         can_auto_refund_sum = sum(proposals.values())
-        can_auto_refund = (allow_partial and can_auto_refund_sum) or can_auto_refund_sum == refund_amount
-    if can_auto_refund:
-        for p, value in proposals.items():
-            with transaction.atomic():
-                r = order.refunds.create(
-                    payment=p,
-                    source=source,
-                    state=OrderRefund.REFUND_STATE_CREATED,
-                    amount=value,
-                    comment=comment,
-                    provider=p.provider
-                )
-                order.log_action('pretix.event.order.refund.created', {
-                    'local_id': r.local_id,
-                    'provider': r.provider,
-                })
-
-            try:
-                r.payment_provider.execute_refund(r)
-            except PaymentException as e:
+        if (allow_partial and can_auto_refund_sum) or can_auto_refund_sum == refund_amount:
+            for p, value in proposals.items():
                 with transaction.atomic():
-                    r.state = OrderRefund.REFUND_STATE_FAILED
-                    r.save()
-                    order.log_action('pretix.event.order.refund.failed', {
+                    r = order.refunds.create(
+                        payment=p,
+                        source=source,
+                        state=OrderRefund.REFUND_STATE_CREATED,
+                        amount=value,
+                        comment=comment,
+                        provider=p.provider
+                    )
+                    order.log_action('pretix.event.order.refund.created', {
                         'local_id': r.local_id,
                         'provider': r.provider,
-                        'error': str(e)
                     })
-                error = True
-                notify_admin = True
-            else:
-                if r.state not in (OrderRefund.REFUND_STATE_TRANSIT, OrderRefund.REFUND_STATE_DONE):
+
+                try:
+                    r.payment_provider.execute_refund(r)
+                except PaymentException as e:
+                    with transaction.atomic():
+                        r.state = OrderRefund.REFUND_STATE_FAILED
+                        r.save()
+                        order.log_action('pretix.event.order.refund.failed', {
+                            'local_id': r.local_id,
+                            'provider': r.provider,
+                            'error': str(e)
+                        })
+                    error = True
                     notify_admin = True
+                else:
+                    if r.state not in (OrderRefund.REFUND_STATE_TRANSIT, OrderRefund.REFUND_STATE_DONE):
+                        notify_admin = True
 
     if refund_amount - can_auto_refund_sum > Decimal('0.00'):
         if manual_refund:
@@ -2500,15 +2506,15 @@ def _try_auto_refund(order, manual_refund=False, allow_partial=False, source=Ord
 @app.task(base=ProfiledTask, bind=True, max_retries=5, default_retry_delay=1, throws=(OrderError,))
 @scopes_disabled()
 def cancel_order(self, order: int, user: int=None, send_mail: bool=True, api_token=None, oauth_application=None,
-                 device=None, cancellation_fee=None, try_auto_refund=False, refund_as_giftcard=False, comment=None,
-                 cancel_invoice=True):
+                 device=None, cancellation_fee=None, try_auto_refund=False, refund_as_giftcard=False,
+                 email_comment=None, refund_comment=None, cancel_invoice=True):
     try:
         try:
             ret = _cancel_order(order, user, send_mail, api_token, device, oauth_application,
-                                cancellation_fee, cancel_invoice=cancel_invoice)
+                                cancellation_fee, cancel_invoice=cancel_invoice, comment=email_comment)
             if try_auto_refund:
                 _try_auto_refund(order, refund_as_giftcard=refund_as_giftcard,
-                                 comment=comment)
+                                 comment=refund_comment)
             return ret
         except LockTimeoutException:
             self.retry()

src/pretix/base/services/seating.py

@@ -45,7 +45,7 @@ def validate_plan_change(event, subevent, plan):
                 seat=OuterRef('pk'),
                 canceled=False,
             ).exclude(
-                order__status=(Order.STATUS_CANCELED, Order.STATUS_EXPIRED)
+                order__status__in=(Order.STATUS_CANCELED, Order.STATUS_EXPIRED)
             ))
         ).annotate(has_v=Count('vouchers')).filter(
             subevent=subevent,
@@ -69,7 +69,7 @@ def generate_seats(event, subevent, plan, mapping, blocked_guids=None):
             seat=OuterRef('pk'),
             canceled=False,
         ).exclude(
-            order__status=Order.STATUS_CANCELED
+            order__status__in=(Order.STATUS_CANCELED, Order.STATUS_EXPIRED)
         )),
         has_v=Count('vouchers')
     ).filter(subevent=subevent).order_by():
@@ -134,7 +134,7 @@ def generate_seats(event, subevent, plan, mapping, blocked_guids=None):
     Seat.objects.bulk_create(create_seats)
     CartPosition.objects.filter(seat__in=[s.pk for s in current_seats.values()]).delete()
     OrderPosition.all.filter(
-        Q(canceled=True) | Q(order__status=Order.STATUS_CANCELED),
+        Q(canceled=True) | Q(order__status__in=(Order.STATUS_CANCELED, Order.STATUS_EXPIRED)),
         seat__in=[s.pk for s in current_seats.values()],
     ).update(seat=None)
     Seat.objects.filter(pk__in=[s.pk for s in current_seats.values()]).delete()

src/pretix/base/settings.py

@@ -86,14 +86,32 @@ def primary_font_kwargs():
     from pretix.presale.style import get_fonts
 
     choices = [('Open Sans', 'Open Sans')]
-    choices += [
-        (a, {"title": a, "data": v}) for a, v in get_fonts().items()
-    ]
+    choices += sorted([
+        (a, {"title": a, "data": v}) for a, v in get_fonts().items() if not v.get('pdf_only', False)
+    ], key=lambda a: a[0])
     return {
         'choices': choices,
     }
 
 
+def restricted_plugin_kwargs():
+    from pretix.base.plugins import get_all_plugins
+
+    plugins_available = [
+        (p.module, p.name) for p in get_all_plugins(None)
+        if (
+            not p.name.startswith('.') and
+            getattr(p, 'restricted', False) and
+            not hasattr(p, 'is_available')  # this means you should not really use restricted and is_available
+        )
+    ]
+    return {
+        'widget': forms.CheckboxSelectMultiple,
+        'label': _("Allow usage of restricted plugins"),
+        'choices': plugins_available,
+    }
+
+
 class LazyI18nStringList(UserList):
     def __init__(self, init_list=None):
         super().__init__()
@@ -109,6 +127,13 @@ class LazyI18nStringList(UserList):
 
 
 DEFAULTS = {
+    'allowed_restricted_plugins': {
+        'default': [],
+        'type': list,
+        'form_class': forms.MultipleChoiceField,
+        'serializer_class': serializers.MultipleChoiceField,
+        'form_kwargs': lambda: restricted_plugin_kwargs(),
+    },
     'customer_accounts': {
         'default': 'False',
         'type': bool,
@@ -1170,7 +1195,20 @@ DEFAULTS = {
             help_text=_("If you ask for a phone number, explain why you do so and what you will use the phone number for.")
         )
     },
-
+    'show_checkin_number_user': {
+        'default': 'False',
+        'type': bool,
+        'serializer_class': serializers.BooleanField,
+        'form_class': forms.BooleanField,
+        'form_kwargs': dict(
+            label=_("Show number of check-ins to customer"),
+            help_text=_('With this option enabled, your customers will be able how many times they entered '
+                        'the event. This is usually not necessary, but might be useful in combination with tickets '
+                        'that are usable a specific number of times, so customers can see how many times they have '
+                        'already been used. Exits or failed scans will not be counted, and the user will not see '
+                        'the different check-in lists.'),
+        )
+    },
     'ticket_download': {
         'default': 'False',
         'type': bool,
@@ -1573,6 +1611,32 @@ DEFAULTS = {
             help_text=_("If enabled, we will attach an .ics calendar file to order confirmation emails."),
         )
     },
+    'mail_attach_ical_paid_only': {
+        'default': 'False',
+        'type': bool,
+        'form_class': forms.BooleanField,
+        'serializer_class': serializers.BooleanField,
+        'form_kwargs': dict(
+            label=_("Attach calendar files only after order has been paid"),
+            help_text=_("Use this if you e.g. put a private access link into the calendar file to make sure people only "
+                        "receive it after their payment was confirmed."),
+        )
+    },
+    'mail_attach_ical_description': {
+        'default': '',
+        'type': LazyI18nString,
+        'form_class': I18nFormField,
+        'form_kwargs': dict(
+            label=_("Event description"),
+            widget=I18nTextarea,
+            help_text=_(
+                "You can use this to share information with your attendees, such as travel information or the link to a digital event. "
+                "If you keep it empty, we will put a link to the event shop, the admission time, and your organizer name in there. "
+                "We do not allow using placeholders with sensitive person-specific data as calendar entries are often shared with an "
+                "unspecified number of people."
+            ),
+        )
+    },
     'mail_prefix': {
         'default': None,
         'type': str,
@@ -1850,6 +1914,8 @@ Your {event} team"""))
 
 your order {code} for {event} has been canceled.
 
+{comment}
+
 You can view the details of your order at
 {url}
 

src/pretix/base/shredder.py

@@ -34,7 +34,6 @@
 
 import json
 import os
-from datetime import timedelta
 from typing import List, Tuple
 
 from django.db import transaction
@@ -70,11 +69,11 @@ def shred_constraints(event: Event):
             max_fromto=Greatest(Max('date_to'), Max('date_from'))
         )
         max_date = max_date['max_fromto'] or max_date['max_to'] or max_date['max_from']
-        if max_date is not None and max_date > now() - timedelta(days=30):
-            return _('Your event needs to be over for at least 30 days to use this feature.')
+        if max_date is not None and max_date >= now():
+            return _('Your event needs to be over to use this feature.')
     else:
-        if (event.date_to or event.date_from) > now() - timedelta(days=30):
-            return _('Your event needs to be over for at least 30 days to use this feature.')
+        if (event.date_to or event.date_from) >= now():
+            return _('Your event needs to be over to use this feature.')
     if event.live:
         return _('Your ticket shop needs to be offline to use this feature.')
     return None

src/pretix/base/templates/pretixbase/email/order_details.html

@@ -90,13 +90,16 @@
                         {% for groupkey, positions in cart %}
                             <tr>
                                 <td>
-                                    {% if not groupkey.4 %} {# is addon #}
+                                    {% if not groupkey.4 %} {# is not addon #}
                                         {{ positions|length }}x
                                     {% endif %}
                                 </td>
                                 <td>
                                     {% if groupkey.4 %} {# is addon #}
                                         +
+                                        {% if positions|length > 1 %}
+                                            {{ positions|length }}x
+                                        {% endif %}
                                     {% endif %}
                                     {{ groupkey.0.name }}{% if groupkey.1 %} – {{ groupkey.1.value }}{% endif %}
                                     {% if groupkey.2 %} {# subevent #}

src/pretix/base/views/mixins.py

@@ -58,7 +58,7 @@ class BaseQuestionsViewMixin:
     def _positions_for_questions(self):
         raise NotImplementedError()
 
-    def get_question_override_sets(self, position):
+    def get_question_override_sets(self, position, index):
         return []
 
     def question_form_kwargs(self, cr):
@@ -72,7 +72,7 @@ class BaseQuestionsViewMixin:
         submitted at once.
         """
         formlist = []
-        for cr in self._positions_for_questions:
+        for idx, cr in enumerate(self._positions_for_questions):
             cartpos = cr if isinstance(cr, CartPosition) else None
             orderpos = cr if isinstance(cr, OrderPosition) else None
 
@@ -96,7 +96,7 @@ class BaseQuestionsViewMixin:
                 ))
             )
 
-            override_sets = self.get_question_override_sets(cr)
+            override_sets = self.get_question_override_sets(cr, idx)
             for overrides in override_sets:
                 for question_name, question_field in form.fields.items():
                     if hasattr(question_field, 'question'):

src/pretix/base/views/redirect.py

@@ -23,8 +23,8 @@ import urllib.parse
 
 from django.core import signing
 from django.http import HttpResponseBadRequest, HttpResponseRedirect
-from django.urls import reverse
 from django.shortcuts import render
+from django.urls import reverse
 
 
 def _is_samesite_referer(request):

src/pretix/base/views/tasks.py

@@ -205,6 +205,8 @@ class AsyncFormView(AsyncMixin, FormView):
     Also, all form keyword arguments except ``instance`` need to be serializable.
     """
     known_errortypes = ['ValidationError']
+    expected_exceptions = (ValidationError,)
+    task_base = ProfiledEventTask
 
     def __init_subclass__(cls):
         def async_execute(self, *, request_path, query_string, form_kwargs, locale, tz, organizer=None, event=None, user=None, session_key=None):
@@ -222,7 +224,7 @@ class AsyncFormView(AsyncMixin, FormView):
             elif organizer:
                 view_instance.request.organizer = organizer
             if user:
-                view_instance.request.user = User.objects.get(pk=user)
+                view_instance.request.user = User.objects.get(pk=user) if isinstance(user, int) else user
             if session_key:
                 engine = import_module(settings.SESSION_ENGINE)
                 self.SessionStore = engine.SessionStore
@@ -231,7 +233,7 @@ class AsyncFormView(AsyncMixin, FormView):
             with translation.override(locale), timezone.override(pytz.timezone(tz)):
                 form_class = view_instance.get_form_class()
                 if form_kwargs.get('instance'):
-                    cls.model.objects.get(pk=form_kwargs['instance'])
+                    form_kwargs['instance'] = cls.model.objects.get(pk=form_kwargs['instance'])
 
                 form_kwargs = view_instance.get_async_form_kwargs(form_kwargs, organizer, event)
                 form = form_class(**form_kwargs)
@@ -239,10 +241,10 @@ class AsyncFormView(AsyncMixin, FormView):
                 return view_instance.async_form_valid(self, form)
 
         cls.async_execute = app.task(
-            base=ProfiledEventTask,
+            base=cls.task_base,
             bind=True,
             name=cls.__module__ + '.' + cls.__name__ + '.async_execute',
-            throws=(ValidationError,)
+            throws=cls.expected_exceptions
         )(async_execute)
 
     def async_form_valid(self, task, form):

src/pretix/control/forms/event.py

@@ -523,6 +523,7 @@ class EventSettingsForm(SettingsForm):
         'last_order_modification_date',
         'allow_modifications_after_checkin',
         'checkout_show_copy_answers_button',
+        'show_checkin_number_user',
         'primary_color',
         'theme_color_success',
         'theme_color_danger',
@@ -872,6 +873,8 @@ class MailSettingsForm(SettingsForm):
         'mail_attach_ical',
         'mail_attach_tickets',
         'mail_attachment_new_order',
+        'mail_attach_ical_paid_only',
+        'mail_attach_ical_description',
     ]
 
     mail_sales_channel_placed_paid = forms.MultipleChoiceField(
@@ -1072,14 +1075,15 @@ class MailSettingsForm(SettingsForm):
         'mail_text_order_free': ['event', 'order'],
         'mail_text_order_free_attendee': ['event', 'order', 'position'],
         'mail_text_order_changed': ['event', 'order'],
-        'mail_text_order_canceled': ['event', 'order'],
+        'mail_text_order_canceled': ['event', 'order', 'comment'],
         'mail_text_order_expire_warning': ['event', 'order'],
         'mail_text_order_custom_mail': ['event', 'order'],
         'mail_text_download_reminder': ['event', 'order'],
         'mail_text_download_reminder_attendee': ['event', 'order', 'position'],
         'mail_text_resend_link': ['event', 'order'],
         'mail_text_waiting_list': ['event', 'waiting_list_entry'],
-        'mail_text_resend_all_links': ['event', 'orders']
+        'mail_text_resend_all_links': ['event', 'orders'],
+        'mail_attach_ical_description': ['event', 'event_or_subevent'],
     }
 
     def _set_field_placeholders(self, fn, base_parameters):

src/pretix/control/forms/filter.py

@@ -255,7 +255,6 @@ class OrderFilterForm(FilterForm):
                 | Q(pk__in=matching_invoices)
                 | Q(pk__in=matching_positions)
                 | Q(pk__in=matching_invoice_addresses)
-                | Q(pk__in=matching_invoices)
             )
             for recv, q in order_search_filter_q.send(sender=getattr(self, 'event', None), query=u):
                 mainq = mainq | q
@@ -1860,11 +1859,11 @@ class VoucherFilterForm(FilterForm):
         for i in self.event.items.prefetch_related('variations').all():
             variations = list(i.variations.all())
             if variations:
-                choices.append((str(i.pk), _('{product} – Any variation').format(product=i.name)))
+                choices.append((str(i.pk), _('{product} – Any variation').format(product=str(i))))
                 for v in variations:
-                    choices.append(('%d-%d' % (i.pk, v.pk), '%s – %s' % (i.name, v.value)))
+                    choices.append(('%d-%d' % (i.pk, v.pk), '%s – %s' % (str(i), v.value)))
             else:
-                choices.append((str(i.pk), i.name))
+                choices.append((str(i.pk), str(i)))
         for q in self.event.quotas.all():
             choices.append(('q-%d' % q.pk, _('Any product in quota "{quota}"').format(quota=q)))
         self.fields['itemvar'].choices = choices
@@ -1881,7 +1880,7 @@ class VoucherFilterForm(FilterForm):
             if s == '<>':
                 qs = qs.filter(Q(tag__isnull=True) | Q(tag=''))
             elif s[0] == '"' and s[-1] == '"':
-                qs = qs.filter(tag__iexact=s[1:-1])
+                qs = qs.filter(tag__exact=s[1:-1])
             else:
                 qs = qs.filter(tag__icontains=s)
 
@@ -2121,7 +2120,7 @@ class CheckinFilterForm(FilterForm):
         self.event = kwargs.pop('event')
         super().__init__(*args, **kwargs)
 
-        self.fields['device'].queryset = self.event.organizer.devices.all()
+        self.fields['device'].queryset = self.event.organizer.devices.all().order_by('device_id')
         self.fields['gate'].queryset = self.event.organizer.gates.all()
 
         self.fields['checkin_list'].queryset = self.event.checkin_lists.all()
@@ -2142,11 +2141,11 @@ class CheckinFilterForm(FilterForm):
         for i in self.event.items.prefetch_related('variations').all():
             variations = list(i.variations.all())
             if variations:
-                choices.append((str(i.pk), _('{product} – Any variation').format(product=i.name)))
+                choices.append((str(i.pk), _('{product} – Any variation').format(product=str(i))))
                 for v in variations:
-                    choices.append(('%d-%d' % (i.pk, v.pk), '%s – %s' % (i.name, v.value)))
+                    choices.append(('%d-%d' % (i.pk, v.pk), '%s – %s' % (str(i), v.value)))
             else:
-                choices.append((str(i.pk), i.name))
+                choices.append((str(i.pk), str(i)))
         self.fields['itemvar'].choices = choices
 
     def filter_qs(self, qs):

src/pretix/control/forms/item.py

@@ -434,6 +434,9 @@ class ItemCreateForm(I18nModelForm):
         if self.cleaned_data.get('copy_from'):
             for question in self.cleaned_data['copy_from'].questions.all():
                 question.items.add(instance)
+                question.log_action('pretix.event.question.changed', user=self.user, data={
+                    'item_added': self.instance.pk
+                })
             for a in self.cleaned_data['copy_from'].addons.all():
                 instance.addons.create(addon_category=a.addon_category, min_count=a.min_count, max_count=a.max_count,
                                        price_included=a.price_included, position=a.position,
@@ -563,7 +566,7 @@ class ItemUpdateForm(I18nModelForm):
             if d['tax_rule'] and d['tax_rule'].rate > 0:
                 self.add_error(
                     'tax_rule',
-                    _("Gift card products should not be associated with non-zero tax rates since sales tax will be applied when the gift card is redeemed.")
+                    _("Gift card products should use a tax rule with a rate of 0 percent since sales tax will be applied when the gift card is redeemed.")
                 )
             if d['admission']:
                 self.add_error(
@@ -627,7 +630,9 @@ class ItemUpdateForm(I18nModelForm):
                 'class': 'scrolling-multiple-choice'
             }),
             'generate_tickets': TicketNullBooleanSelect(),
-            'show_quota_left': ShowQuotaNullBooleanSelect()
+            'show_quota_left': ShowQuotaNullBooleanSelect(),
+            'max_per_order': forms.widgets.NumberInput(attrs={'min': 0}),
+            'min_per_order': forms.widgets.NumberInput(attrs={'min': 0}),
         }
 
 

src/pretix/control/forms/orders.py

@@ -167,6 +167,12 @@ class CancelForm(ForceQuotaConfirmationForm):
         initial=True,
         required=False
     )
+    comment = forms.CharField(
+        label=_('Comment (will be sent to the user)'),
+        help_text=_('Will be included in the notification email when the respective placeholder is present in the '
+                    'configured email text.'),
+        required=False,
+    )
 
     def __init__(self, *args, **kwargs):
         super().__init__(*args, **kwargs)
@@ -482,6 +488,9 @@ class OrderPositionChangeForm(forms.Form):
         self.fields['tax_rule'].queryset = instance.event.tax_rules.all()
         self.fields['tax_rule'].label_from_instance = self.taxrule_label_from_instance
 
+        if instance.addon_to_id:
+            del self.fields['operation_split']
+
         if not instance.seat and not (
                 instance.item.seat_category_mappings.filter(subevent=instance.subevent).exists()
         ):
@@ -612,7 +621,7 @@ class OrderMailForm(forms.Form):
     )
     attach_tickets = forms.BooleanField(
         label=_("Attach tickets"),
-        help_text=_("Will be ignored if all tickets in this order exceed a given size limit to ensure email deliverability."),
+        help_text=_("Will be ignored if tickets exceed a given size limit to ensure email deliverability."),
         required=False
     )
     attach_invoices = forms.ModelMultipleChoiceField(
@@ -746,16 +755,17 @@ class EventCancelForm(forms.Form):
     auto_refund = forms.BooleanField(
         label=_('Automatically refund money if possible'),
         initial=True,
-        required=False
+        required=False,
+        help_text=_('Only available for payment method that support automatic refunds.')
     )
     manual_refund = forms.BooleanField(
-        label=_('Create manual refund if the payment method does not support automatic refunds'),
-        widget=forms.CheckboxInput(attrs={'data-display-dependency': '#id_auto_refund'}),
+        label=_('Create refund in the manual refund to-do list'),
         initial=True,
         required=False,
-        help_text=_('If checked, all payments with a payment method not supporting automatic refunds will be on your '
-                    'manual refund to-do list. Do not check if you want to refund some of the orders by offsetting '
-                    'with different orders or issuing gift cards.')
+        help_text=_('Manual refunds will be created which will be listed in the manual refund to-do list. '
+                    'When combined with the automatic refund functionally, only payments with a payment method not '
+                    'supporting automatic refunds will be on your manual refund to-do list. Do not check if you want '
+                    'to refund some of the orders by offsetting with different orders or issuing gift cards.')
     )
     refund_as_giftcard = forms.BooleanField(
         label=_('Refund order value to a gift card instead instead of the original payment method'),
@@ -840,7 +850,7 @@ class EventCancelForm(forms.Form):
             label=_("Subject"),
             required=True,
             widget_kwargs={'attrs': {'data-display-dependency': '#id_send'}},
-            initial=_('Canceled: {event}'),
+            initial=LazyI18nString.from_gettext(gettext_noop('Canceled: {event}')),
             widget=I18nTextInput,
             locales=self.event.settings.get('locales'),
         )
@@ -866,7 +876,7 @@ class EventCancelForm(forms.Form):
         self.fields['send_waitinglist_subject'] = I18nFormField(
             label=_("Subject"),
             required=True,
-            initial=_('Canceled: {event}'),
+            initial=LazyI18nString.from_gettext(gettext_noop('Canceled: {event}')),
             widget=I18nTextInput,
             widget_kwargs={'attrs': {'data-display-dependency': '#id_send_waitinglist'}},
             locales=self.event.settings.get('locales'),

src/pretix/control/forms/organizer.py

@@ -39,6 +39,7 @@ from django import forms
 from django.conf import settings
 from django.core.exceptions import ValidationError
 from django.db.models import Q
+from django.forms.utils import ErrorDict
 from django.utils.crypto import get_random_string
 from django.utils.safestring import mark_safe
 from django.utils.translation import gettext_lazy as _, pgettext_lazy
@@ -268,6 +269,69 @@ class DeviceForm(forms.ModelForm):
         }
 
 
+class DeviceBulkEditForm(forms.ModelForm):
+
+    def __init__(self, *args, **kwargs):
+        organizer = kwargs.pop('organizer')
+        self.mixed_values = kwargs.pop('mixed_values')
+        self.queryset = kwargs.pop('queryset')
+        super().__init__(*args, **kwargs)
+        self.fields['limit_events'].queryset = organizer.events.all().order_by(
+            '-has_subevents', '-date_from'
+        )
+        self.fields['gate'].queryset = organizer.gates.all()
+
+    def clean(self):
+        d = super().clean()
+        if self.prefix + '__events' in self.data.getlist('_bulk') and not d['all_events'] and not d['limit_events']:
+            raise ValidationError(_('Your device will not have access to anything, please select some events.'))
+
+        return d
+
+    class Meta:
+        model = Device
+        fields = ['all_events', 'limit_events', 'security_profile', 'gate']
+        widgets = {
+            'limit_events': forms.CheckboxSelectMultiple(attrs={
+                'data-inverse-dependency': '#id_all_events',
+                'class': 'scrolling-multiple-choice scrolling-multiple-choice-large',
+            }),
+        }
+        field_classes = {
+            'limit_events': SafeEventMultipleChoiceField
+        }
+
+    def save(self, commit=True):
+        objs = list(self.queryset)
+        fields = set()
+
+        check_map = {
+            'all_events': '__events',
+            'limit_events': '__events',
+        }
+        for k in self.fields:
+            cb_val = self.prefix + check_map.get(k, k)
+            if cb_val not in self.data.getlist('_bulk'):
+                continue
+
+            fields.add(k)
+            for obj in objs:
+                if k == 'limit_events':
+                    getattr(obj, k).set(self.cleaned_data[k])
+                else:
+                    setattr(obj, k, self.cleaned_data[k])
+
+        if fields:
+            Device.objects.bulk_update(objs, [f for f in fields if f != 'limit_events'], 200)
+
+    def full_clean(self):
+        if len(self.data) == 0:
+            # form wasn't submitted
+            self._errors = ErrorDict()
+            return
+        super().full_clean()
+
+
 class OrganizerSettingsForm(SettingsForm):
     timezone = forms.ChoiceField(
         choices=((a, a) for a in common_timezones),
@@ -286,6 +350,7 @@ class OrganizerSettingsForm(SettingsForm):
         required=False,
     )
     auto_fields = [
+        'allowed_restricted_plugins',
         'customer_accounts',
         'customer_accounts_link_by_email',
         'invoice_regenerate_allowed',
@@ -339,7 +404,12 @@ class OrganizerSettingsForm(SettingsForm):
     )
 
     def __init__(self, *args, **kwargs):
+        is_admin = kwargs.pop('is_admin', False)
         super().__init__(*args, **kwargs)
+
+        if not is_admin:
+            del self.fields['allowed_restricted_plugins']
+
         self.fields['name_scheme'].choices = (
             (k, _('Ask for {fields}, display like {example}').format(
                 fields=' + '.join(str(vv[1]) for vv in v['fields']),
@@ -422,6 +492,7 @@ class MailSettingsForm(SettingsForm):
                 if f == 'full_name':
                     continue
                 placeholders['name_%s' % f] = name_scheme['sample'][f]
+            placeholders['name_for_salutation'] = _("Mr Doe")
         return placeholders
 
     def _set_field_placeholders(self, fn, base_parameters):

src/pretix/control/forms/vouchers.py

@@ -385,6 +385,12 @@ class VoucherBulkForm(VoucherForm):
         if vouchers.exists():
             raise ValidationError(_('A voucher with one of these codes already exists.'))
 
+        codes_seen = set()
+        for c in data['codes']:
+            if c in codes_seen:
+                raise ValidationError(_('The voucher code {code} appears in your list twice.').format(code=c))
+            codes_seen.add(c)
+
         if data.get('send') and not all([data.get('send_subject'), data.get('send_message'), data.get('send_recipients')]):
             raise ValidationError(_('If vouchers should be sent by email, subject, message and recipients need to be specified.'))
 

src/pretix/control/logdisplay.py

@@ -341,13 +341,12 @@ def pretixcontrol_logentry_display(sender: Event, logentry: LogEntry, **kwargs):
         'pretix.event.order.paid': _('The order has been marked as paid.'),
         'pretix.event.order.cancellationrequest.deleted': _('The cancellation request has been deleted.'),
         'pretix.event.order.refunded': _('The order has been refunded.'),
-        'pretix.event.order.canceled': _('The order has been canceled.'),
         'pretix.event.order.reactivated': _('The order has been reactivated.'),
         'pretix.event.order.deleted': _('The test mode order {code} has been deleted.'),
         'pretix.event.order.placed': _('The order has been created.'),
         'pretix.event.order.placed.require_approval': _('The order requires approval before it can continue to be processed.'),
         'pretix.event.order.approved': _('The order has been approved.'),
-        'pretix.event.order.denied': _('The order has been denied.'),
+        'pretix.event.order.denied': _('The order has been denied (comment: "{comment}").'),
         'pretix.event.order.contact.changed': _('The email address has been changed from "{old_email}" '
                                                 'to "{new_email}".'),
         'pretix.event.order.contact.confirmed': _('The email address has been confirmed to be working (the user clicked on a link '
@@ -423,6 +422,7 @@ def pretixcontrol_logentry_display(sender: Event, logentry: LogEntry, **kwargs):
         'pretix.voucher.added': _('The voucher has been created.'),
         'pretix.voucher.sent': _('The voucher has been sent to {recipient}.'),
         'pretix.voucher.added.waitinglist': _('The voucher has been created and sent to a person on the waiting list.'),
+        'pretix.voucher.expired.waitinglist': _('The voucher has been set to expire because the recipient removed themselves from the waiting list.'),
         'pretix.voucher.changed': _('The voucher has been changed.'),
         'pretix.voucher.deleted': _('The voucher has been deleted.'),
         'pretix.voucher.redeemed': _('The voucher has been redeemed in order {order_code}.'),
@@ -531,6 +531,27 @@ def pretixcontrol_logentry_display(sender: Event, logentry: LogEntry, **kwargs):
             bleach.clean(logentry.parsed_data.get('msg'), tags=[], strip=True)
         )
 
+    if logentry.action_type == 'pretix.event.order.canceled':
+        comment = logentry.parsed_data.get('comment')
+        if comment:
+            return _('The order has been canceled (comment: "{comment}").').format(comment=comment)
+        else:
+            return _('The order has been canceled.')
+
+    if logentry.action_type in ('pretix.control.views.checkin.reverted', 'pretix.event.checkin.reverted'):
+        if 'list' in data:
+            try:
+                checkin_list = sender.checkin_lists.get(pk=data.get('list')).name
+            except CheckinList.DoesNotExist:
+                checkin_list = _("(unknown)")
+        else:
+            checkin_list = _("(unknown)")
+
+        return _('The check-in of position #{posid} on list "{list}" has been reverted.').format(
+            posid=data.get('positionid'),
+            list=checkin_list,
+        )
+
     if sender and logentry.action_type.startswith('pretix.event.checkin'):
         return _display_checkin(sender, logentry)
 
@@ -559,20 +580,6 @@ def pretixcontrol_logentry_display(sender: Event, logentry: LogEntry, **kwargs):
             list=checkin_list
         )
 
-    if logentry.action_type in ('pretix.control.views.checkin.reverted', 'pretix.event.checkin.reverted'):
-        if 'list' in data:
-            try:
-                checkin_list = sender.checkin_lists.get(pk=data.get('list')).name
-            except CheckinList.DoesNotExist:
-                checkin_list = _("(unknown)")
-        else:
-            checkin_list = _("(unknown)")
-
-        return _('The check-in of position #{posid} on list "{list}" has been reverted.').format(
-            posid=data.get('positionid'),
-            list=checkin_list,
-        )
-
     if logentry.action_type == 'pretix.team.member.added':
         return _('{user} has been added to the team.').format(user=data.get('email'))
 

src/pretix/control/middleware.py

@@ -142,6 +142,18 @@ class PermissionMiddleware:
             return redirect(reverse('control:user.settings.2fa'))
 
         if 'event' in url.kwargs and 'organizer' in url.kwargs:
+            if url.kwargs['organizer'] == '-' and url.kwargs['event'] == '-':
+                # This is a hack that just takes the user to ANY event. It's useful to link to features in support
+                # or documentation.
+                ev = request.user.get_events_with_any_permission().order_by('-date_from').first()
+                if not ev:
+                    raise Http404(_("The selected event was not found or you "
+                                    "have no permission to administrate it."))
+                k = dict(url.kwargs)
+                k['organizer'] = ev.organizer.slug
+                k['event'] = ev.slug
+                return redirect(reverse(url.view_name, kwargs=k, args=url.args))
+
             with scope(organizer=None):
                 request.event = Event.objects.filter(
                     slug=url.kwargs['event'],
@@ -157,6 +169,17 @@ class PermissionMiddleware:
             else:
                 request.eventpermset = request.user.get_event_permission_set(request.organizer, request.event)
         elif 'organizer' in url.kwargs:
+            if url.kwargs['organizer'] == '-':
+                # This is a hack that just takes the user to ANY organizer. It's useful to link to features in support
+                # or documentation.
+                org = request.user.get_organizers_with_any_permission().first()
+                if not org:
+                    raise Http404(_("The selected organizer was not found or you "
+                                    "have no permission to administrate it."))
+                k = dict(url.kwargs)
+                k['organizer'] = org.slug
+                return redirect(reverse(url.view_name, kwargs=k, args=url.args))
+
             request.organizer = Organizer.objects.filter(
                 slug=url.kwargs['organizer'],
             ).first()

src/pretix/control/templates/pretixcontrol/boxoffice/payment.html

@@ -57,7 +57,11 @@
         {% endif %}
     {% elif payment_info.payment_type == "izettle" %}
         <dt>{% trans "Payment provider" %}</dt>
-        <dd>iZettle</dd>
+        <dd>Zettle</dd>
+        {% if payment_info.payment_data.reference %}
+            <dt>{% trans "Paymeant reference" %}</dt>
+            <dd>{{ payment_info.payment_data.reference }}</dd>
+        {% endif %}
         <dt>{% trans "Payment Application" %}</dt>
         <dd>{{ payment_info.payment_data.applicationName }}</dd>
         <dt>{% trans "Card Entry Mode" %}</dt>

src/pretix/control/templates/pretixcontrol/checkin/list_edit.html

@@ -93,6 +93,17 @@
                         </div>
 
                     </div>
+                    <div class="alert alert-info" v-if="missingItems.length">
+                        <p>
+                            {% trans "Your rule always filters by product or variation, but the following products or variations are not contained in any of your rule parts so people with these tickets will not get in:" %}
+                        </p>
+                        <ul>
+                            <li v-for="h in missingItems">{{ "{" }}{h}{{ "}" }}</li>
+                        </ul>
+                        <p>
+                            {% trans "Please double-check if this was intentional." %}
+                        </p>
+                    </div>
                 </div>
                 <div class="disabled-withoutjs sr-only">
                     {{ form.rules }}
@@ -105,6 +116,7 @@
             </button>
         </div>
     </form>
+    {{ items|json_script:"items" }}
 
     {% compress js %}
         <script type="text/javascript" src="{% static "vuejs/vue.js" %}"></script>
@@ -118,6 +130,7 @@
         <script type="text/javascript" src="{% static "d3/d3-transition.v2.js" %}"></script>
         <script type="text/javascript" src="{% static "d3/d3-drag.v2.js" %}"></script>
         <script type="text/javascript" src="{% static "d3/d3-zoom.v2.js" %}"></script>
+        <script type="text/javascript" src="{% static "pretixcontrol/js/ui/checkinrules/jsonlogic-boolalg.js" %}"></script>
         <script type="text/vue" src="{% static 'pretixcontrol/js/ui/checkinrules/datetimefield.vue' %}"></script>
         <script type="text/vue" src="{% static 'pretixcontrol/js/ui/checkinrules/timefield.vue' %}"></script>
         <script type="text/vue" src="{% static 'pretixcontrol/js/ui/checkinrules/lookup-select2.vue' %}"></script>

src/pretix/control/templates/pretixcontrol/dashboard.html

@@ -56,7 +56,7 @@
             {% endfor %}
         </div>
         <p class="">
-            <a href="{% url "control:events" %}?ordering=date_from&status=-date_to" class="">
+            <a href="{% url "control:events" %}?ordering=date_from&status=date_past" class="">
                 {% trans "View all recent events" %}
             </a>
         </p>

src/pretix/control/templates/pretixcontrol/event/fragment_plugin_description.html

@@ -0,0 +1,36 @@
+{% load i18n %}
+{% if show_meta %}
+    {% if plugin.author %}
+        <p class="meta text-muted">
+            {% blocktrans trimmed with a=plugin.author %}
+                by <em>{{ a }}</em>
+            {% endblocktrans %}</p>
+    {% endif %}
+{% endif %}
+<p>{{ plugin.description|safe }}</p>
+{% if plugin.restricted and plugin.module not in request.event.settings.allowed_restricted_plugins %}
+    <p class="text-muted">
+        <span class="fa fa-info-circle" aria-hidden="true"></span>
+        {% trans "This plugin needs to be enabled by a system administrator for your account." %}
+    </p>
+{% endif %}
+{% if plugin.app.compatibility_errors %}
+    <div class="alert alert-warning">
+        {% trans "This plugin cannot be enabled for the following reasons:" %}
+        <ul>
+            {% for e in plugin.app.compatibility_errors %}
+                <li>{{ e }}</li>
+            {% endfor %}
+        </ul>
+    </div>
+{% endif %}
+{% if plugin.app.compatibility_warnings %}
+    <div class="alert alert-warning">
+        {% trans "This plugin reports the following problems:" %}
+        <ul>
+            {% for e in plugin.app.compatibility_warnings %}
+                <li>{{ e }}</li>
+            {% endfor %}
+        </ul>
+    </div>
+{% endif %}

src/pretix/control/templates/pretixcontrol/event/mail.html

@@ -14,7 +14,6 @@
                 <legend>{% trans "General" %}</legend>
                 {% bootstrap_field form.mail_prefix layout="control" %}
                 {% bootstrap_field form.mail_attach_tickets layout="control" %}
-                {% bootstrap_field form.mail_attach_ical layout="control" %}
                 {% url "control:organizer.settings.mail" organizer=request.organizer.slug as org_url %}
                 {% propagated request.event org_url "mail_from" "smtp_use_custom" "smtp_host" "smtp_port" "smtp_username" "smtp_password" "smtp_use_tls" "smtp_use_ssl" %}
                     <div class="form-group">
@@ -57,6 +56,12 @@
                 {% endpropagated %}
                 {% bootstrap_field form.mail_sales_channel_placed_paid layout="control" %}
             </fieldset>
+            <fieldset>
+                <legend>{% trans "Calendar invites" %}</legend>
+                {% bootstrap_field form.mail_attach_ical layout="control" %}
+                {% bootstrap_field form.mail_attach_ical_paid_only layout="control" %}
+                {% bootstrap_field form.mail_attach_ical_description layout="control" %}
+            </fieldset>
             <fieldset>
                 <legend>{% trans "E-mail design" %}</legend>
                 <div class="row">

src/pretix/control/templates/pretixcontrol/event/plugins.html

@@ -1,8 +1,15 @@
 {% extends "pretixcontrol/event/settings_base.html" %}
 {% load i18n %}
+{% load static %}
 {% load bootstrap3 %}
 {% block inside %}
-    <h1>{% trans "Installed plugins" %}</h1>
+    <h1>{% trans "Available plugins" %}</h1>
+    <p>
+        {% blocktrans trimmed %}
+            On this page, you can choose plugins you want to enable for your event. Plugins might bring additional
+            software functionality, connect your event to third-party services, or apply other forms of customizations.
+        {% endblocktrans %}
+    </p>
     <form action="" method="post" class="form-horizontal form-plugins">
         {% csrf_token %}
         {% if "success" in request.GET %}
@@ -11,71 +18,71 @@
             </div>
         {% endif %}
         <div class="tabbed-form">
-            {% for cat, catlabel, plist in plugins %}
+            {% for cat, catlabel, plist, has_pictures in plugins %}
                 <fieldset>
                     <legend>{{ catlabel }}</legend>
-                    <div class="table-responsive">
-                        <table class="table">
-                            {% for plugin in plist %}
-                                <tr class="{% if plugin.app.compatibility_errors %}warning{% elif plugin.module in plugins_active %}success{% else %}default{% endif %}">
-                                    <td>
-                                        <strong>{{ plugin.name }}</strong>
-                                        {% if plugin.author %}
-                                            <p class="meta text-muted">
-                                                {% blocktrans trimmed with v=plugin.version a=plugin.author %}
-                                                    Version {{ v }} by <em>{{ a }}</em>
-                                                {% endblocktrans %}</p>
-                                        {% else %}
-                                            <p class="meta text-muted">
-                                                {% blocktrans trimmed with v=plugin.version a=plugin.author %}
-                                                    Version {{ v }}
-                                                {% endblocktrans %}</p>
+                    <div class="plugin-list">
+                        {% for plugin in plist %}
+                            <div class="plugin-container {% if plugin.featured %}featured-plugin{% endif %}">
+                            {% if plugin.featured %}
+                               <div class="panel panel-default">
+                                    <div class="panel-body">
+                            {% endif %}
+                                        <div class="plugin-text">
+                                        {% if plugin.featured or plugin.experimental %}
+                                            <p class="text-muted">
+                                            {% if plugin.featured %}
+                                                <span class="fa fa-thumbs-up" aria-hidden="true"></span>
+                                                {% trans "Top recommendation" %}
+                                            {% endif %}
+                                            {% if plugin.experimental %}
+                                                <span class="fa fa-flask" aria-hidden="true"></span>
+                                                {% trans "Experimental feature" %}
+                                            {% endif %}
+                                            </p>
                                         {% endif %}
-                                        <p>{{ plugin.description }}</p>
-                                        {% if plugin.restricted and not request.user.is_staff %}
-                                            <span class="text-muted">
-                                                {% trans "This plugin needs to be enabled by a system administrator for your event." %}
-                                            </span>
+                                        {% if plugin.picture %}
+                                            <p><img src="{% static plugin.picture %}" class="plugin-picture"></p>
                                         {% endif %}
+                                            <h4>
+                                                {{ plugin.name }}
+                                                {% if show_meta %}
+                                                    <span class="text-muted text-sm">{{ plugin.version }}</span>
+                                                {% endif %}
+                                                {% if plugin.module in plugins_active %}
+                                                    <span class="label label-success">
+                                                        <span class="fa fa-check" aria-hidden="true"></span>
+                                                        {% trans "Active" %}
+                                                    </span>
+                                                {% endif %}
+                                            </h4>
+                                            {% include "pretixcontrol/event/fragment_plugin_description.html" with plugin=plugin %}
+                                        </div>
                                         {% if plugin.app.compatibility_errors %}
-                                            <div class="alert alert-warning">
-                                                {% trans "This plugin cannot be enabled for the following reasons:" %}
-                                                <ul>
-                                                    {% for e in plugin.app.compatibility_errors %}
-                                                        <li>{{ e }}</li>
-                                                    {% endfor %}
-                                                </ul>
+                                            <div class="plugin-action">
+                                                <span class="text-muted">{% trans "Incompatible" %}</span>
+                                            </div>
+                                        {% elif plugin.restricted and plugin.module not in request.event.settings.allowed_restricted_plugins %}
+                                            <div class="plugin-action">
+                                                <span class="text-muted">{% trans "Not available" %}</span>
+                                            </div>
+                                        {% elif plugin.module in plugins_active %}
+                                            <div class="plugin-action flip">
+                                                <button class="btn btn-default{% if plugin.featured %} btn-lg{% endif %}" name="plugin:{{ plugin.module }}"
+                                                        value="disable">{% trans "Disable" %}</button>
+                                            </div>
+                                        {% else %}
+                                            <div class="plugin-action flip">
+                                                <button class="btn btn-primary{% if plugin.featured %} btn-lg{% endif %}" name="plugin:{{ plugin.module }}"
+                                                        value="enable">{% trans "Enable" %}</button>
                                             </div>
                                         {% endif %}
-                                        {% if plugin.app.compatibility_warnings %}
-                                            <div class="alert alert-warning">
-                                                {% trans "This plugin reports the following problems:" %}
-                                                <ul>
-                                                    {% for e in plugin.app.compatibility_warnings %}
-                                                        <li>{{ e }}</li>
-                                                    {% endfor %}
-                                                </ul>
-                                            </div>
-                                        {% endif %}
-                                    </td>
-                                    <td class="text-right flip" width="20%">
-                                        {% if plugin.app.compatibility_errors %}
-                                            <button class="btn disabled btn-block btn-default"
-                                                    disabled="disabled">{% trans "Incompatible" %}</button>
-                                            {% elif plugin.restricted and not staff_session %}
-                                            <button class="btn disabled btn-block btn-default"
-                                                    disabled="disabled">{% trans "Not available" %}</button>
-                                            {% elif plugin.module in plugins_active %}
-                                            <button class="btn btn-default btn-block" name="plugin:{{ plugin.module }}"
-                                                    value="disable">{% trans "Disable" %}</button>
-                                        {% else %}
-                                            <button class="btn btn-default btn-block" name="plugin:{{ plugin.module }}"
-                                                    value="enable">{% trans "Enable" %}</button>
-                                        {% endif %}
-                                    </td>
-                                </tr>
-                            {% endfor %}
-                        </table>
+                            {% if plugin.featured %}
+                                    </div>
+                                </div>
+                            {% endif %}
+                            </div>
+                        {% endfor %}
                     </div>
                 </fieldset>
             {% endfor %}

src/pretix/control/templates/pretixcontrol/event/settings.html

@@ -221,6 +221,7 @@
                 {% bootstrap_field sform.show_items_outside_presale_period layout="control" %}
                 {% bootstrap_field sform.last_order_modification_date layout="control" %}
                 {% bootstrap_field sform.allow_modifications_after_checkin layout="control" %}
+                {% bootstrap_field sform.show_checkin_number_user layout="control" %}
             </fieldset>
             <fieldset>
                 <legend>{% trans "Display" %}</legend>

src/pretix/control/templates/pretixcontrol/item/index.html

@@ -20,6 +20,11 @@
                         <div class="form-group">
                             <label class="col-md-3 control-label">{% trans "Product type" %}</label>
                             <div class="col-md-9">
+                                {% for e in form.errors.admission %}
+                                    <div class="alert alert-danger has-error">
+                                        {{ e }}
+                                    </div>
+                                {% endfor %}
                                 <div class="big-radio radio">
                                     <label>
                                         <input type="radio" value="on" name="{{ form.admission.html_name }}" {% if form.admission.value %}checked{% endif %}>

src/pretix/control/templates/pretixcontrol/items/index.html

@@ -46,7 +46,7 @@
                 {% for c in cat_list %}
                     <tbody data-dnd-url="{% url "control:event.items.reorder" organizer=request.event.organizer.slug event=request.event.slug %}">
                     {% for i in c.list %}
-                        {% if forloop.counter0 == 0 and i.category %}<tr class="sortable-disabled"><th colspan="8" scope="colgroup" class="text-muted">{{ i.category.name }}</th></tr>{% endif %}
+                        {% if forloop.counter0 == 0 and i.category %}<tr class="sortable-disabled"><th colspan="8" scope="colgroup" class="text-muted">{{ i.category }}</th></tr>{% endif %}
                         <tr data-dnd-id="{{ i.id }}" {% if not i.active %}class="row-muted"{% endif %}>
                             <td><strong>
                                 {% if not i.active %}<strike>{% endif %}

src/pretix/control/templates/pretixcontrol/items/quota_delete.html

@@ -13,7 +13,7 @@
                 <ul>
                     {% for item in dependent %}
                         <li>
-                            <a href="{% url "control:event.item" organizer=request.event.organizer.slug event=request.event.slug item=item.pk %}">{{ item.name }}</a>
+                            <a href="{% url "control:event.item" organizer=request.event.organizer.slug event=request.event.slug item=item.pk %}">{{ item }}</a>
                         </li>
                     {% endfor %}
                 </ul>

src/pretix/control/templates/pretixcontrol/order/cancel.html

@@ -1,6 +1,7 @@
 {% extends "pretixcontrol/event/base.html" %}
 {% load i18n %}
 {% load bootstrap3 %}
+{% load money %}
 {% block title %}
     {% trans "Cancel order" %}
 {% endblock %}
@@ -22,13 +23,22 @@
         {% csrf_token %}
         <input type="hidden" name="status" value="c"/>
         {% bootstrap_form_errors form %}
+        {% if form.cancellation_fee %}
+            {% if fee %}
+                {% with fee|money:request.event.currency as f %}
+                    <p>{% blocktrans trimmed with fee="<strong>"|add:f|add:"</strong>"|safe %}
+                        The configured cancellation fee for a self-service cancellation would be {{ fee }} for this
+                        order, but for a cancellation performed by you, you need to set the cancellation fee here:
+                    {% endblocktrans %}</p>
+                {% endwith %}
+            {% endif %}
+            {% bootstrap_field form.cancellation_fee layout='' %}
+        {% endif %}
         {% bootstrap_field form.send_email layout='' %}
+        {% bootstrap_field form.comment layout='' %}
         {% if form.cancel_invoice %}
             {% bootstrap_field form.cancel_invoice layout='' %}
         {% endif %}
-        {% if form.cancellation_fee %}
-            {% bootstrap_field form.cancellation_fee layout='' %}
-        {% endif %}
         <div class="row checkout-button-row">
             <div class="col-md-4">
                 <a class="btn btn-block btn-default btn-lg"

src/pretix/control/templates/pretixcontrol/order/change.html

@@ -202,10 +202,12 @@
                         </div>
 
                         {% bootstrap_field position.form.operation_cancel layout='inline' %}
-                        {% bootstrap_field position.form.operation_split layout='inline' %}
+                        {% if position.form.operation_split %}
+                            {% bootstrap_field position.form.operation_split layout='inline' %}
+                        {% endif %}
                         {% if position.addons.exists %}
                             <em class="text-danger">
-                                {% trans "Removing this position will also remove all add-ons to this position." %}
+                                {% trans "Removing or splitting this position will also remove or split all add-ons to this position." %}
                             </em>
                         {% endif %}
                     </div>

src/pretix/control/templates/pretixcontrol/order/refund_choose.html

@@ -8,6 +8,7 @@
 {% block content %}
     <h1>
         {% trans "Refund order" %}
+        <small class="text-muted">{{ full_refund|money:request.event.currency }}</small>
         <a class="btn btn-link btn-lg"
                 href="{% url "control:event.order" event=request.event.slug organizer=request.event.organizer.slug code=order.code %}">
             {% blocktrans trimmed with order=order.code %}

src/pretix/control/templates/pretixcontrol/orders/export.html

@@ -11,7 +11,8 @@
         {% endif %}
     </h1>
     {% for e in exporters %}
-        <details class="panel panel-default" {% if "identifier" in request.GET or "exporter" in request.POST %}open{% endif %}>
+        <details class="panel panel-default"
+            {% if request.GET.identifier == e.identifier or request.POST.exporter == e.identifier %}open{% endif %}>
             <summary class="panel-heading">
                 <h3 class="panel-title">
                     {{ e.verbose_name }}

src/pretix/control/templates/pretixcontrol/organizers/device_bulk_edit.html

@@ -0,0 +1,46 @@
+{% extends "pretixcontrol/organizers/base.html" %}
+{% load i18n %}
+{% load bootstrap3 %}
+{% block inner %}
+    <h1>
+        {% trans "Change multiple devices" %}
+        <small>
+            {% blocktrans trimmed with number=devices.count %}
+                {{ number }} selected
+            {% endblocktrans %}
+        </small>
+    </h1>
+    <form class="form-horizontal" action="" method="post">
+        {% csrf_token %}
+        {% bootstrap_form_errors form %}
+        <div class="hidden">
+            {% for d in devices %}
+                <input type="hidden" name="device" value="{{ d.pk }}">
+            {% endfor %}
+        </div>
+        <fieldset>
+            <legend>{% trans "General" %}</legend>
+            <div class="bulk-edit-field-group">
+                <label class="field-toggle">
+                    <input type="checkbox" name="_bulk" value="{{ form.prefix }}__events" {% if form.prefix|add:"__events" in bulk_selected %}checked{% endif %}>
+                    {% trans "change" context "form_bulk" %}
+                </label>
+                <div class="field-content">
+                    {% bootstrap_field form.all_events layout="control" %}
+                    {% bootstrap_field form.limit_events layout="control" %}
+                </div>
+            </div>
+        </fieldset>
+        <p>&nbsp;</p>
+        <fieldset>
+            <legend>{% trans "Advanced settings" %}</legend>
+            {% bootstrap_field form.security_profile layout="bulkedit" %}
+            {% bootstrap_field form.gate layout="bulkedit" %}
+        </fieldset>
+        <div class="form-group submit-group">
+            <button type="submit" class="btn btn-primary btn-save">
+                {% trans "Save" %}
+            </button>
+        </div>
+    </form>
+{% endblock %}

src/pretix/control/templates/pretixcontrol/organizers/devices.html

@@ -21,7 +21,7 @@
             </p>
 
             <a href="{% url "control:organizer.device.add" organizer=request.organizer.slug %}"
-                    class="btn btn-primary btn-lg"><i class="fa fa-plus"></i> {% trans "Connect a device" %}</a>
+               class="btn btn-primary btn-lg"><i class="fa fa-plus"></i> {% trans "Connect a device" %}</a>
         </div>
     {% else %}
         <div class="panel panel-default">
@@ -53,101 +53,139 @@
         </div>
         <p>
             <a href="{% url "control:organizer.device.add" organizer=request.organizer.slug %}"
-                    class="btn btn-default"><i class="fa fa-plus"></i> {% trans "Connect a device" %}</a>
+               class="btn btn-default"><i class="fa fa-plus"></i> {% trans "Connect a device" %}</a>
         </p>
-        <div class="table-responsive">
-            <table class="table table-condensed table-hover">
-                <thead>
-                <tr>
-                    <th>{% trans "Device ID" %}
-                        <a href="?{% url_replace request 'ordering' '-device_id' %}"><i class="fa fa-caret-down"></i></a>
-                        <a href="?{% url_replace request 'ordering' 'device_id' %}"><i class="fa fa-caret-up"></i></a></th>
-                    </th>
-                    <th>{% trans "Name" %}
-                        <a href="?{% url_replace request 'ordering' '-name' %}"><i class="fa fa-caret-down"></i></a>
-                        <a href="?{% url_replace request 'ordering' 'name' %}"><i class="fa fa-caret-up"></i></a></th>
-                    </th>
-                    <th>{% trans "Hardware model" %}</th>
-                    <th>{% trans "Software" %}</th>
-                    <th>{% trans "Setup date" %}
-                        <a href="?{% url_replace request 'ordering' '-initialized' %}"><i class="fa fa-caret-down"></i></a>
-                        <a href="?{% url_replace request 'ordering' 'initialized' %}"><i class="fa fa-caret-up"></i></a></th>
-                    </th>
-                    <th>{% trans "Events" %}</th>
-                    <th></th>
-                </tr>
-                </thead>
-                <tbody>
-                {% for d in devices %}
-                    <tr {% if d.revoked %}class="text-muted"{% endif %}>
-                        <td>
-                            {{ d.device_id }}
-                        </td>
-                        <td>
-                            {% if d.revoked %}<del>{% endif %}
-                            {{ d.name }}
-                            {% if d.revoked %}</del>{% endif %}
-                            {% if d.gate %}
-                                <br>
-                                <small class="text-muted">{{ d.gate.name }}</small>
-                            {% endif %}
-                            <br>
-                            <small class="text-muted">{{ d.unique_serial }}</small>
-                        </td>
-                        <td>
-                            {{ d.hardware_brand|default_if_none:"" }} {{ d.hardware_model|default_if_none:"" }}
-                        </td>
-                        <td>
-                            {{ d.software_brand|default_if_none:"" }} {{ d.software_version|default_if_none:"" }}
-                        </td>
-                        <td>
-                            {% if d.initialized %}
-                                {{ d.initialized|date:"SHORT_DATETIME_FORMAT" }}
-                            {% else %}
-                                <em>{% trans "Not yet initialized" %}</em>
-                            {% endif %}
-                            {% if d.revoked %}
-                                <span class="label label-danger">{% trans "Revoked" %}</span>
-                            {% endif %}
-                        </td>
-                        <td>
-                            {% if d.all_events %}
-                                {% trans "All" %}
-                            {% else %}
-                                <ul>
-                                    {% for e in d.limit_events.all %}
-                                        <li>
-                                            <a href="{% url "control:event.index" organizer=request.organizer.slug event=e.slug %}">
-                                                {{ e }}
-                                            </a>
-                                        </li>
-                                    {% endfor %}
-                                </ul>
-                            {% endif %}
-                        </td>
-                        <td class="text-right flip">
-                            {% if not d.initialized %}
-                                <a href="{% url "control:organizer.device.connect" organizer=request.organizer.slug device=d.id %}"
-                                        class="btn btn-primary btn-sm"><i class="fa fa-link"></i>
-                                    {% trans "Connect" %}</a>
-                            {% elif d.api_token %}
-                                <a href="{% url "control:organizer.device.revoke" organizer=request.organizer.slug device=d.id %}"
-                                        class="btn btn-default btn-sm">
-                                    {% trans "Revoke access" %}</a>
-                            {% endif %}
-                            <a href="{% url "control:organizer.device.logs" organizer=request.organizer.slug device=d.id %}"
-                               class="btn btn-default btn-sm">
-                                <span class="fa fa-list-alt"></span>
-                                {% trans "Logs" %}
-                            </a>
-                            <a href="{% url "control:organizer.device.edit" organizer=request.organizer.slug device=d.id %}"
-                                    class="btn btn-default btn-sm"><i class="fa fa-edit"></i></a>
-                        </td>
+        <form action="{% url "control:organizer.device.bulk_edit" organizer=request.organizer.slug %}" method="post">
+            {% csrf_token %}
+            <div class="hidden">
+                {{ filter_form.as_p }}
+            </div>
+            <div class="table-responsive">
+                <table class="table table-condensed table-hover table-quotas">
+                    <thead>
+                    <tr>
+                        <th>
+                            <label aria-label="{% trans "select all rows for batch-operation" %}"
+                                   class="batch-select-label"><input type="checkbox" data-toggle-table/></label>
+                        </th>
+                        <th>{% trans "Device ID" %}
+                            <a href="?{% url_replace request 'ordering' '-device_id' %}"><i
+                                    class="fa fa-caret-down"></i></a>
+                            <a href="?{% url_replace request 'ordering' 'device_id' %}"><i
+                                    class="fa fa-caret-up"></i></a>
+                        </th>
+                        <th>{% trans "Name" %}
+                            <a href="?{% url_replace request 'ordering' '-name' %}"><i class="fa fa-caret-down"></i></a>
+                            <a href="?{% url_replace request 'ordering' 'name' %}"><i class="fa fa-caret-up"></i></a>
+                        </th>
+                        <th>{% trans "Hardware model" %}</th>
+                        <th>{% trans "Software" %}</th>
+                        <th>{% trans "Setup date" %}
+                            <a href="?{% url_replace request 'ordering' '-initialized' %}"><i
+                                    class="fa fa-caret-down"></i></a>
+                            <a href="?{% url_replace request 'ordering' 'initialized' %}"><i class="fa fa-caret-up"></i></a>
+                        </th>
+                        <th>{% trans "Events" %}</th>
+                        <th></th>
                     </tr>
-                {% endfor %}
-                </tbody>
-            </table>
-        </div>
+                    {% if page_obj.paginator.num_pages > 1 %}
+                        <tr class="table-select-all warning hidden">
+                            <td>
+                                <input type="checkbox" name="__ALL" id="__all"
+                                       data-results-total="{{ page_obj.paginator.count }}">
+                            </td>
+                            <td colspan="7">
+                                <label for="__all">
+                                    {% trans "Select all results on other pages as well" %}
+                                </label>
+                            </td>
+                        </tr>
+                    {% endif %}
+                    </thead>
+                    <tbody>
+                    {% for d in devices %}
+                        <tr {% if d.revoked %}class="text-muted"{% endif %}>
+                            <td>
+                                <label aria-label="{% trans "select row for batch-operation" %}"
+                                       class="batch-select-label"><input type="checkbox" name="device"
+                                                                         class="batch-select-checkbox"
+                                                                         value="{{ d.pk }}"/></label>
+                            </td>
+                            <td>
+                                {{ d.device_id }}
+                            </td>
+                            <td>
+                                {% if d.revoked %}
+                                    <del>{% endif %}
+                                {{ d.name }}
+                                {% if d.revoked %}</del>{% endif %}
+                                {% if d.gate %}
+                                    <br>
+                                    <small class="text-muted">{{ d.gate.name }}</small>
+                                {% endif %}
+                                <br>
+                                <small class="text-muted">{{ d.unique_serial }}</small>
+                            </td>
+                            <td>
+                                {{ d.hardware_brand|default_if_none:"" }} {{ d.hardware_model|default_if_none:"" }}
+                            </td>
+                            <td>
+                                {{ d.software_brand|default_if_none:"" }} {{ d.software_version|default_if_none:"" }}
+                            </td>
+                            <td>
+                                {% if d.initialized %}
+                                    {{ d.initialized|date:"SHORT_DATETIME_FORMAT" }}
+                                {% else %}
+                                    <em>{% trans "Not yet initialized" %}</em>
+                                {% endif %}
+                                {% if d.revoked %}
+                                    <span class="label label-danger">{% trans "Revoked" %}</span>
+                                {% endif %}
+                            </td>
+                            <td>
+                                {% if d.all_events %}
+                                    {% trans "All" %}
+                                {% else %}
+                                    <ul>
+                                        {% for e in d.limit_events.all %}
+                                            <li>
+                                                <a href="{% url "control:event.index" organizer=request.organizer.slug event=e.slug %}">
+                                                    {{ e }}
+                                                </a>
+                                            </li>
+                                        {% endfor %}
+                                    </ul>
+                                {% endif %}
+                            </td>
+                            <td class="text-right flip">
+                                {% if not d.initialized %}
+                                    <a href="{% url "control:organizer.device.connect" organizer=request.organizer.slug device=d.id %}"
+                                       class="btn btn-primary btn-sm"><i class="fa fa-link"></i>
+                                        {% trans "Connect" %}</a>
+                                    {% elif d.api_token %}
+                                    <a href="{% url "control:organizer.device.revoke" organizer=request.organizer.slug device=d.id %}"
+                                       class="btn btn-default btn-sm">
+                                        {% trans "Revoke access" %}</a>
+                                {% endif %}
+                                <a href="{% url "control:organizer.device.logs" organizer=request.organizer.slug device=d.id %}"
+                                   class="btn btn-default btn-sm">
+                                    <span class="fa fa-list-alt"></span>
+                                    {% trans "Logs" %}
+                                </a>
+                                <a href="{% url "control:organizer.device.edit" organizer=request.organizer.slug device=d.id %}"
+                                   class="btn btn-default btn-sm"><i class="fa fa-edit"></i></a>
+                            </td>
+                        </tr>
+                    {% endfor %}
+                    </tbody>
+                </table>
+            </div>
+            <div class="batch-select-actions">
+                <button type="submit" class="btn btn-primary btn-save" name="action" value="edit">
+                    <i class="fa fa-edit"></i>{% trans "Edit selected" %}
+                </button>
+            </div>
+        </form>
         {% include "pretixcontrol/pagination.html" %}
     {% endif %}
 {% endblock %}

src/pretix/control/templates/pretixcontrol/organizers/edit.html

@@ -36,6 +36,9 @@
                         {% bootstrap_field sform.contact_mail layout="control" %}
                         {% bootstrap_field sform.organizer_info_text layout="control" %}
                         {% bootstrap_field sform.event_team_provisioning layout="control" %}
+                        {% if sform.allowed_restricted_plugins %}
+                            {% bootstrap_field sform.allowed_restricted_plugins layout="control" %}
+                        {% endif %}
                     </fieldset>
                     <fieldset>
                         <legend>{% trans "Organizer page" %}</legend>

src/pretix/control/templates/pretixcontrol/pdf/index.html

@@ -23,7 +23,7 @@
     </script>
     <div class="row">
         <div class="col-md-9">
-            <div class="panel panel-default">
+            <div class="panel panel-default panel-pdf-editor">
                 <div class="panel-heading">
                     <div class="pull-right flip">
                         <div class="btn-group">
@@ -48,6 +48,8 @@
                     {% trans "Editor" %}
                 </div>
                 <div class="panel-body">
+                    <ul class="nav nav-pills" id="page_nav">
+                    </ul>
                     <div id="editor-canvas-area">
                         <canvas id="pdf-canvas"
                                 data-pdf-url="{{ pdf }}"
@@ -193,7 +195,7 @@
                             <span class="btn btn-default fileinput-button background-button">
                                 <i class="fa fa-upload"></i>
                                 <span>{% trans "Upload custom background" %}</span>
-                                <input id="fileupload" type="file" name="background">
+                                <input id="fileupload" type="file" name="background" accept="application/pdf">
                             </span>
                         </div>
                         <div class="col-sm-12 help-inline">
@@ -204,6 +206,14 @@
                                 {% endblocktrans %}
                             </p>
                         </div>
+                        <div class="col-sm-12">
+                            <p>
+                                <a class="btn btn-default background-download-button" href="{{ pdf }}" target="_blank">
+                                    <i class="fa fa-download"></i>
+                                    <span>{% trans "Download current background" %}</span>
+                                </a>
+                            </p>
+                        </div>
                     </div>
                     <div class="row control-group pdf-info">
                         <div class="col-sm-12">
@@ -357,12 +367,14 @@
                             </select>
                         </div>
                     </div>
-                    <div class="row control-group text">
+                    <div class="row control-group text textcontent">
                         <div class="col-sm-12">
-                            <label>{% trans "Text content" %}</label><br>
+                            <label>{% trans "Content" %}</label><br>
                             <select class="input-block-level form-control" id="toolbox-content">
                                 {% for varname, var in variables.items %}
-                                    <option data-sample="{{ var.editor_sample }}" value="{{ varname }}">{{ var.label }}</option>
+                                    {% if not var.hidden %}
+                                        <option data-sample="{{ var.editor_sample }}" {% if var.migrate_from %}data-old-value="{{ var.migrate_from }}"{% endif %} value="{{ varname }}">{{ var.label }}</option>
+                                    {% endif %}
                                 {% endfor %}
                                 {% for p in request.organizer.meta_properties.all %}
                                     <option value="meta:{{ p.name }}">
@@ -374,10 +386,19 @@
                                         {% trans "Item attribute:" %} {{ p.name }}
                                     </option>
                                 {% endfor %}
+                                <option value="other_i18n">{% trans "Other… (multilingual)" %}</option>
                                 <option value="other">{% trans "Other…" %}</option>
                             </select>
                             <textarea type="text" value="" class="input-block-level form-control"
                                     id="toolbox-content-other"></textarea>
+                            <div class="i18n-form-group" id="toolbox-content-other-i18n">
+                                {% for l in request.event.settings.locales %}
+                                    <textarea id="toolbox-content-other-{{ l }}" rows="3" class="input-block-level form-control" title="{{ l }}" lang="{{ l }}"></textarea>
+                                {% endfor %}
+                            </div>
+                            <p class="help-block" id="toolbox-content-other-help">
+                                <a href="?placeholders=true" target="_blank">{% trans "Show available placeholders" %}</a>
+                            </p>
                         </div>
                     </div>
                 </div>
@@ -401,13 +422,20 @@
                         <span class="fa fa-qrcode"></span>
                         {% trans "QR code for Lead Scanning" %}
                     </button>
+                    <button class="btn btn-default btn-block" id="editor-add-qrcode-other"
+                            data-content="secret"
+                            disabled>
+                        <span class="fa fa-qrcode"></span>
+                        {% trans "Other QR code" %}
+                    </button>
                     <button class="btn btn-default btn-block" id="editor-add-poweredby"
                             data-content="dark"
                             disabled>
                         <span class="fa fa-image"></span>
                         {% trans "pretix Logo" %}
                     </button>
-                    <button class="btn btn-default btn-block" id="editor-add-image" disabled>
+                    <button class="btn btn-default btn-block" id="editor-add-image" disabled
+                        data-toggle="tooltip" title="{% trans "You can use this to add user-uploaded pictures from questions or pictures generated by plugins. If you want to embed a logo or other images, use a custom background instead." %}">
                         <span class="fa fa-image"></span>
                         {% trans "Dynamic image" %}
                     </button>

src/pretix/control/templates/pretixcontrol/pdf/placeholders.html

@@ -0,0 +1,68 @@
+{% extends "pretixcontrol/event/base.html" %}
+{% load i18n %}
+{% load static %}
+{% load compress %}
+{% block title %}{% trans "PDF Editor" %}{% endblock %}
+{% block custom_header %}
+    {{ block.super }}
+    {% compress css %}
+        <link type="text/css" rel="stylesheet" href="{% static "pretixcontrol/scss/pdfeditor.css" %}">
+    {% endcompress %}
+    <link type="text/css" rel="stylesheet" href="{% url "control:pdf.css" %}">
+{% endblock %}
+{% block content %}
+    <h1>
+        {% trans "PDF Editor" %}
+        <small>{% trans "Available placeholders" %}</small>
+    </h1>
+    <p>
+        {% blocktrans trimmed %}
+            You can use placeholders in custom texts on tickets to enrich your text with individual data. Which
+            placeholders are available depends on your event settings, activated plugins, the selected product,
+            as well as user input.
+            This page lists all placeholders technically available for your event, however most of them can also
+            be empty in some cases depending on configuration.
+        {% endblocktrans %}
+    </p>
+    <div class="table-responsive">
+        <table class="table table-hover table-condensed">
+            <thead>
+            <tr>
+                <th>{% trans "Placeholder" %}</th>
+                <th>{% trans "Description" %}</th>
+                <th>{% trans "Formatting example" %}</th>
+            </tr>
+            </thead>
+            <tbody>
+            {% for varname, var in variables.items %}
+                {% if not var.hidden %}
+                    <tr>
+                        <td><code>{{ "{" }}{{ varname }}{{ "}" }}</code></td>
+                        <td>{{ var.label }}</td>
+                        <td>{{ var.editor_sample }}</td>
+                    </tr>
+                {% endif %}
+            {% endfor %}
+            {% for p in request.organizer.meta_properties.all %}
+                <tr>
+                    <td><code>{{ "{" }}meta:{{ p.name }}{{ "}" }}</code></td>
+                    <td>
+                        {% trans "Event attribute:" %} {{ p.name }}
+                    </td>
+                    <td></td>
+                </tr>
+            {% endfor %}
+            {% for p in request.event.item_meta_properties.all %}
+                <tr>
+                    <td><code>{{ "{" }}itemmeta:{{ p.name }}{{ "}" }}</code></td>
+                    <td>
+                        {% trans "Item attribute:" %} {{ p.name }}
+                    </td>
+                    <td></td>
+                </tr>
+            {% endfor %}
+            </tbody>
+        </table>
+    </div>
+{% endblock %}
+

src/pretix/control/templates/pretixcontrol/subevents/index.html

@@ -188,7 +188,7 @@
                     <button type="submit" class="btn btn-danger btn-save" name="action" value="delete">
                         <i class="fa fa-trash"></i>{% trans "Delete selected" %}
                     </button>
-                    <button type="submit" class="btn btn-primary btn-save" name="action" value="disable"
+                    <button type="submit" class="btn btn-primary btn-save" name="action" value="edit"
                             formaction="{% url "control:event.subevents.bulkedit"  organizer=request.event.organizer.slug event=request.event.slug %}">
                         <i class="fa fa-edit"></i>{% trans "Edit selected" %}
                     </button>

src/pretix/control/templates/pretixcontrol/users/form.html

@@ -38,6 +38,14 @@
                             <input name="text" value="{{ backend }}" class="form-control" disabled>
                         </div>
                     </div>
+                    {% if user.auth_backend_identifier %}
+                        <div class="form-group">
+                            <label class="col-md-3 control-label">{% trans "External identifier" %}</label>
+                            <div class="col-md-9">
+                                <input name="text" value="{{ user.auth_backend_identifier }}" class="form-control" disabled>
+                            </div>
+                        </div>
+                    {% endif %}
                     {% bootstrap_field form.email layout='control' %}
                     {% if form.new_pw %}
                         {% bootstrap_field form.new_pw layout='control' %}

src/pretix/control/templates/pretixcontrol/vouchers/detail.html

@@ -39,17 +39,15 @@
                     <legend>{% trans "Voucher details" %}</legend>
                     {% bootstrap_field form.code layout="control" %}
                     {% if voucher.pk %}
-                        {% if not request.event.has_subevents or voucher.subevent %}
-                            <div class="form-group">
-                                <label class="col-md-3 control-label" for="id_url">{% trans "Voucher link" %}</label>
-                                <div class="col-md-9">
-                                    <input type="text" name="url"
-                                            value="{% abseventurl request.event "presale:event.redeem" %}?voucher={{ voucher.code|urlencode }}{% if voucher.subevent_id %}&subevent={{ voucher.subevent_id }}{% endif %}"
-                                            class="form-control"
-                                            id="id_url" readonly>
-                                </div>
+                        <div class="form-group">
+                            <label class="col-md-3 control-label" for="id_url">{% trans "Voucher link" %}</label>
+                            <div class="col-md-9">
+                                <input type="text" name="url"
+                                        value="{% abseventurl request.event "presale:event.redeem" %}?voucher={{ voucher.code|urlencode }}{% if voucher.subevent_id %}&subevent={{ voucher.subevent_id }}{% endif %}"
+                                        class="form-control"
+                                        id="id_url" readonly>
                             </div>
-                        {% endif %}
+                        </div>
                     {% endif %}
                     {% bootstrap_field form.max_usages layout="control" %}
                     {% bootstrap_field form.valid_until layout="control" %}

src/pretix/control/templatetags/getitem.py

@@ -29,4 +29,9 @@ def getitem_filter(value, itemname):
     if not value:
         return ''
 
-    return value[itemname]
+    try:
+        return value[itemname]
+    except KeyError:
+        return ''
+    except TypeError:
+        return ''

src/pretix/control/templatetags/hierarkey_form.py

@@ -45,7 +45,8 @@ class PropagatedNode(Node):
             <div class="propagated-settings-box locked panel panel-default">
                 <div class="panel-heading">
                     <input type="hidden" name="_settings_ignore" value="{fnames}">
-                    <button class="btn btn-default pull-right btn-xs" name="decouple" value="{fnames}" data-action="unlink">
+                    <input type="hidden" name="decouple" value="">
+                    <button type="button" class="btn btn-default pull-right btn-xs" value="{fnames}" data-action="unlink">
                         <span class="fa fa-unlock"></span> {text_unlink}
                     </button>
                     <h4 class="panel-title">

src/pretix/control/urls.py

@@ -164,6 +164,8 @@ urlpatterns = [
     re_path(r'^organizer/(?P<organizer>[^/]+)/devices$', organizer.DeviceListView.as_view(), name='organizer.devices'),
     re_path(r'^organizer/(?P<organizer>[^/]+)/device/add$', organizer.DeviceCreateView.as_view(),
             name='organizer.device.add'),
+    re_path(r'^organizer/(?P<organizer>[^/]+)/device/bulk_edit$', organizer.DeviceBulkUpdateView.as_view(),
+            name='organizer.device.bulk_edit'),
     re_path(r'^organizer/(?P<organizer>[^/]+)/device/(?P<device>[^/]+)/edit$', organizer.DeviceUpdateView.as_view(),
             name='organizer.device.edit'),
     re_path(r'^organizer/(?P<organizer>[^/]+)/device/(?P<device>[^/]+)/connect$', organizer.DeviceConnectView.as_view(),