Bump to 4.6.1

doc/admin/installation/docker_smallscale.rst

@@ -36,6 +36,9 @@ Linux and firewalls, we recommend that you start with `ufw`_.
           SSL certificates can be obtained for free these days. We also *do not* provide support for HTTP-only
           installations except for evaluation purposes.
 
+.. warning:: We recommend **PostgreSQL**. If you go for MySQL, make sure you run **MySQL 5.7 or newer** or
+             **MariaDB 10.2.7 or newer**.
+
 .. warning:: By default, using `ufw` in conjunction will not have any effect. Please make sure to either bind the exposed
              ports of your docker container explicitly to 127.0.0.1 or configure docker to respect any set up firewall
              rules.
@@ -58,9 +61,6 @@ directory writable to the user that runs pretix inside the docker container::
 Database
 --------
 
-.. warning:: **Please use PostgreSQL for all new installations**. If you need to go for MySQL, make sure you run
-             **MySQL 5.7 or newer** or **MariaDB 10.2.7 or newer**.
-
 Next, we need a database and a database user. We can create these with any kind of database managing tool or directly on
 our database's shell. Please make sure that UTF8 is used as encoding for the best compatibility. You can check this with
 the following command::
@@ -91,8 +91,6 @@ When using MySQL, make sure you set the character set of the database to ``utf8m
 
     mysql > CREATE DATABASE pretix DEFAULT CHARACTER SET utf8mb4 DEFAULT COLLATE utf8mb4_unicode_ci;
 
-You will also need to make sure that ``sql_mode`` in your ``my.cnf`` file does **not** include ``ONLY_FULL_GROUP_BY``.
-
 Redis
 -----
 

doc/admin/installation/manual_smallscale.rst

@@ -34,6 +34,9 @@ Linux and firewalls, we recommend that you start with `ufw`_.
           SSL certificates can be obtained for free these days. We also *do not* provide support for HTTP-only
           installations except for evaluation purposes.
 
+.. warning:: We recommend **PostgreSQL**. If you go for MySQL, make sure you run **MySQL 5.7 or newer** or
+             **MariaDB 10.2.7 or newer**.
+
 Unix user
 ---------
 
@@ -47,9 +50,6 @@ In this guide, all code lines prepended with a ``#`` symbol are commands that yo
 Database
 --------
 
-.. warning:: **Please use PostgreSQL for all new installations**. If you need to go for MySQL, make sure you run
-             **MySQL 5.7 or newer** or **MariaDB 10.2.7 or newer**.
-
 Having the database server installed, we still need a database and a database user. We can create these with any kind
 of database managing tool or directly on our database's shell. Please make sure that UTF8 is used as encoding for the
 best compatibility. You can check this with the following command::
@@ -65,8 +65,6 @@ When using MySQL, make sure you set the character set of the database to ``utf8m
 
     mysql > CREATE DATABASE pretix DEFAULT CHARACTER SET utf8mb4 DEFAULT COLLATE utf8mb4_unicode_ci;
 
-You will also need to make sure that ``sql_mode`` in your ``my.cnf`` file does **not** include ``ONLY_FULL_GROUP_BY``.
-
 Package dependencies
 --------------------
 

doc/plugins/imported_secrets.rst

@@ -1,301 +0,0 @@
-Secrets Import
-==============
-
-Usually, pretix generates ticket secrets (i.e. the QR code used for scanning) itself. You can read more about this
-process at :ref:`secret_generators`.
-
-With the "Secrets Import" plugin, you can upload your own list of secrets to be used instead. This is useful for
-integrating with third-party check-in systems.
-
-
-API Resource description
--------------------------
-
-The secrets import plugin provides a HTTP API that allows you to create new secrets.
-
-The imported secret resource contains the following public fields:
-
-.. rst-class:: rest-resource-table
-
-===================================== ========================== =======================================================
-Field                                 Type                       Description
-===================================== ========================== =======================================================
-id                                    integer                    Internal ID of the secret
-secret                                string                     Actual string content of the secret (QR code content)
-used                                  boolean                    Whether the secret was already used for a ticket. If ``true``,
-                                                                 the secret can no longer be deleted. Secrets are never used
-                                                                 twice, even if an order is canceled or deleted.
-item                                  integer                    Internal ID of a product, or ``null``. If set, the secret
-                                                                 will only be used for tickets of this product.
-variation                             integer                    Internal ID of a product variation, or ``null``. If set, the secret
-                                                                 will only be used for tickets of this product variation.
-subevent                              integer                    Internal ID of an event series date, or ``null``. If set, the secret
-                                                                 will only be used for tickets of this event series date.
-===================================== ========================== =======================================================
-
-API Endpoints
--------------
-
-.. http:get:: /api/v1/organizers/(organizer)/events/(event)/imported_secrets/
-
-   Returns a list of all secrets imported for an event.
-
-   **Example request**:
-
-   .. sourcecode:: http
-
-      GET /api/v1/organizers/bigevents/events/sampleconf/imported_secrets/ HTTP/1.1
-      Host: pretix.eu
-      Accept: application/json, text/javascript
-
-   **Example response**:
-
-   .. sourcecode:: http
-
-      HTTP/1.1 200 OK
-      Vary: Accept
-      Content-Type: application/json
-
-      {
-        "count": 1,
-        "next": null,
-        "previous": null,
-        "results": [
-          {
-            "id": 1,
-            "secret": "foobar",
-            "used": false,
-            "item": null,
-            "variation": null,
-            "subevent": null
-          }
-        ]
-      }
-
-   :query page: The page number in case of a multi-page result set, default is 1
-   :param organizer: The ``slug`` field of a valid organizer
-   :param event: The ``slug`` field of the event to fetch
-   :statuscode 200: no error
-   :statuscode 401: Authentication failure
-   :statuscode 403: The requested organizer or event does not exist **or** you have no permission to view it.
-
-.. http:get:: /api/v1/organizers/(organizer)/events/(event)/imported_secrets/(id)/
-
-   Returns information on one secret, identified by its ID.
-
-   **Example request**:
-
-   .. sourcecode:: http
-
-      GET /api/v1/organizers/bigevents/events/sampleconf/imported_secrets/1/ HTTP/1.1
-      Host: pretix.eu
-      Accept: application/json, text/javascript
-
-   **Example response**:
-
-   .. sourcecode:: http
-
-      HTTP/1.1 200 OK
-      Vary: Accept
-      Content-Type: application/json
-
-      {
-        "id": 1,
-        "secret": "foobar",
-        "used": false,
-        "item": null,
-        "variation": null,
-        "subevent": null
-      }
-
-   :param organizer: The ``slug`` field of the organizer to fetch
-   :param event: The ``slug`` field of the event to fetch
-   :param id: The ``id`` field of the secret to fetch
-   :statuscode 200: no error
-   :statuscode 401: Authentication failure
-   :statuscode 403: The requested organizer/event/secret does not exist **or** you have no permission to view it.
-
-.. http:post:: /api/v1/organizers/(organizer)/events/(event)/imported_secrets/
-
-   Create a new secret.
-
-   **Example request**:
-
-   .. sourcecode:: http
-
-      POST /api/v1/organizers/bigevents/events/sampleconf/imported_secrets/ HTTP/1.1
-      Host: pretix.eu
-      Accept: application/json, text/javascript
-      Content-Type: application/json
-      Content-Length: 166
-
-      {
-        "secret": "foobar",
-        "used": false,
-        "item": null,
-        "variation": null,
-        "subevent": null
-      }
-
-   **Example response**:
-
-   .. sourcecode:: http
-
-      HTTP/1.1 201 Created
-      Vary: Accept
-      Content-Type: application/json
-
-      {
-        "id": 1,
-        "secret": "foobar",
-        "used": false,
-        "item": null,
-        "variation": null,
-        "subevent": null
-      }
-
-   :param organizer: The ``slug`` field of the organizer to a create new secret for
-   :param event: The ``slug`` field of the event to create a new secret for
-   :statuscode 201: no error
-   :statuscode 400: The secret could not be created due to invalid submitted data.
-   :statuscode 401: Authentication failure
-   :statuscode 403: The requested organizer/event does not exist **or** you have no permission to create secrets.
-
-.. http:post:: /api/v1/organizers/(organizer)/events/(event)/imported_secrets/bulk_create/
-
-   Create new secrets in bulk (up to 500 per request). The request either succeeds or fails entirely.
-
-   **Example request**:
-
-   .. sourcecode:: http
-
-      POST /api/v1/organizers/bigevents/events/sampleconf/imported_secrets/bulk_create/ HTTP/1.1
-      Host: pretix.eu
-      Accept: application/json, text/javascript
-      Content-Type: application/json
-      Content-Length: 166
-
-      [
-        {
-          "secret": "foobar",
-          "used": false,
-          "item": null,
-          "variation": null,
-          "subevent": null
-        },
-        {
-          "secret": "baz",
-          "used": false,
-          "item": null,
-          "variation": null,
-          "subevent": null
-        }
-      ]
-
-   **Example response**:
-
-   .. sourcecode:: http
-
-      HTTP/1.1 200 OK
-      Vary: Accept
-      Content-Type: application/json
-
-      [
-        {
-          "id": 1,
-          "secret": "foobar",
-          "used": false,
-          "item": null,
-          "variation": null,
-          "subevent": null
-        },
-        {
-          "id": 2,
-          "secret": "baz",
-          "used": false,
-          "item": null,
-          "variation": null,
-          "subevent": null
-        }
-      ]
-
-   :param organizer: The ``slug`` field of the organizer to create new secrets for
-   :param event: The ``slug`` field of the event to create new secrets for
-   :statuscode 201: no error
-   :statuscode 400: The secrets could not be created due to invalid submitted data.
-   :statuscode 401: Authentication failure
-   :statuscode 403: The requested organizer/event does not exist **or** you have no permission to create secrets.
-
-
-.. http:patch:: /api/v1/organizers/(organizer)/events/(event)/imported_secrets/(id)/
-
-   Update a secret. You can also use ``PUT`` instead of ``PATCH``. With ``PUT``, you have to provide all fields of
-   the resource, other fields will be reset to default. With ``PATCH``, you only need to provide the fields that you
-   want to change.
-
-   **Example request**:
-
-   .. sourcecode:: http
-
-      PATCH /api/v1/organizers/bigevents/events/sampleconf/imported_secrets/1/ HTTP/1.1
-      Host: pretix.eu
-      Accept: application/json, text/javascript
-      Content-Type: application/json
-      Content-Length: 34
-
-      {
-        "item": 2
-      }
-
-   **Example response**:
-
-   .. sourcecode:: http
-
-      HTTP/1.1 200 OK
-      Vary: Accept
-      Content-Type: text/javascript
-
-      {
-        "id": 1,
-        "secret": "foobar",
-        "used": false,
-        "item": 2,
-        "variation": null,
-        "subevent": null
-      }
-
-   :param organizer: The ``slug`` field of the organizer to modify
-   :param event: The ``slug`` field of the event to modify
-   :param id: The ``id`` field of the secret to modify
-   :statuscode 200: no error
-   :statuscode 400: The secret could not be modified due to invalid submitted data.
-   :statuscode 401: Authentication failure
-   :statuscode 403: The requested organizer/event/secret does not exist **or** you have no permission to change it.
-
-
-.. http:delete:: /api/v1/organizers/(organizer)/events/(event)/imported_secrets/(id)/
-
-   Delete a secret. You can only delete secrets that have not yet been used.
-
-   **Example request**:
-
-   .. sourcecode:: http
-
-      DELETE /api/v1/organizers/bigevents/events/sampleconf/imported_secrets/1/ HTTP/1.1
-      Host: pretix.eu
-      Accept: application/json, text/javascript
-
-   **Example response**:
-
-   .. sourcecode:: http
-
-      HTTP/1.1 204 No Content
-      Vary: Accept
-
-   :param organizer: The ``slug`` field of the organizer to modify
-   :param event: The ``slug`` field of the event to modify
-   :param id: The ``id`` field of the secret to delete
-   :statuscode 204: no error
-   :statuscode 401: Authentication failure
-   :statuscode 403: The requested organizer/event/secret does not exist **or** you have no permission to change it **or** the secret has already been used
-

doc/plugins/index.rst

@@ -17,5 +17,4 @@ If you want to **create** a plugin, please go to the
    campaigns
    certificates
    digital
-   imported_secrets
    webinar

doc/user/events/ticket_secrets.rst

@@ -1,5 +1,3 @@
-.. _secret_generators:
-
 Ticket secret generators
 ========================
 

src/pretix/__init__.py

@@ -19,4 +19,4 @@
 # You should have received a copy of the GNU Affero General Public License along with this program.  If not, see
 # <https://www.gnu.org/licenses/>.
 #
-__version__ = "4.7.0.dev0"
+__version__ = "4.6.1"

src/pretix/api/views/order.py

@@ -659,13 +659,12 @@ class OrderViewSet(viewsets.ModelViewSet):
 
                 _order_placed_email(
                     request.event, order, payment.payment_provider if payment else None, email_template,
-                    log_entry, invoice, payment, is_free=free_flow
+                    log_entry, invoice, payment
                 )
                 if email_attendees:
                     for p in order.positions.all():
                         if p.addon_to_id is None and p.attendee_email and p.attendee_email != order.email:
-                            _order_placed_email_attendee(request.event, order, p, email_attendees_template, log_entry,
-                                                         is_free=free_flow)
+                            _order_placed_email_attendee(request.event, order, p, email_attendees_template, log_entry)
 
                 if not free_flow and order.status == Order.STATUS_PAID and payment:
                     payment._send_paid_mail(invoice, None, '')

src/pretix/base/email.py

@@ -33,7 +33,6 @@ from django.core.mail.backends.smtp import EmailBackend
 from django.db.models import Count
 from django.dispatch import receiver
 from django.template.loader import get_template
-from django.utils.formats import date_format
 from django.utils.timezone import now
 from django.utils.translation import (
     get_language, gettext_lazy as _, pgettext_lazy,
@@ -165,20 +164,9 @@ class TemplateBasedMailRenderer(BaseHTMLMailRenderer):
                 has_addons=Count('addons')
             ))
             htmlctx['cart'] = [(k, list(v)) for k, v in groupby(
-                sorted(
-                    positions,
-                    key=lambda op: (
-                        (op.addon_to.positionid if op.addon_to_id else op.positionid),
-                        op.positionid
-                    )
-                ),
-                key=lambda op: (
-                    op.item,
-                    op.variation,
-                    op.subevent,
-                    op.attendee_name,
-                    (op.pk if op.addon_to_id else None),
-                    (op.pk if op.has_addons else None)
+                positions, key=lambda op: (
+                    op.item, op.variation, op.subevent, op.attendee_name,
+                    (op.pk if op.addon_to_id else None), (op.pk if op.has_addons else None)
                 )
             )]
 
@@ -465,15 +453,6 @@ def base_placeholders(sender, **kwargs):
                 }
             ),
         ),
-        SimpleFunctionalMailTextPlaceholder(
-            'event_location', ['event_or_subevent'], lambda event_or_subevent: str(event_or_subevent.location or ''),
-            lambda event: str(event.location or ''),
-        ),
-        SimpleFunctionalMailTextPlaceholder(
-            'event_admission_time', ['event_or_subevent'],
-            lambda event_or_subevent: date_format(event_or_subevent.date_admission, 'TIME_FORMAT') if event_or_subevent.date_admission else '',
-            lambda event: date_format(event.date_admission, 'TIME_FORMAT') if event.date_admission else '',
-        ),
         SimpleFunctionalMailTextPlaceholder(
             'subevent', ['waiting_list_entry', 'event'],
             lambda waiting_list_entry, event: str(waiting_list_entry.subevent or event),
@@ -643,10 +622,6 @@ def base_placeholders(sender, **kwargs):
             'meta_%s' % k, ['event'], lambda event, k=k: event.meta_data[k],
             v
         ))
-        ph.append(SimpleFunctionalMailTextPlaceholder(
-            'meta_%s' % k, ['event_or_subevent'], lambda event_or_subevent, k=k: event_or_subevent.meta_data[k],
-            v
-        ))
 
     return ph
 

src/pretix/base/exporter.py

@@ -33,7 +33,6 @@
 # License for the specific language governing permissions and limitations under the License.
 
 import io
-import re
 import tempfile
 from collections import OrderedDict, namedtuple
 from decimal import Decimal
@@ -46,26 +45,13 @@ from django.conf import settings
 from django.db.models import QuerySet
 from django.utils.formats import localize
 from django.utils.translation import gettext, gettext_lazy as _
-from openpyxl import Workbook
-from openpyxl.cell.cell import ILLEGAL_CHARACTERS_RE, KNOWN_TYPES, Cell
 
 from pretix.base.models import Event
+from pretix.helpers.safe_openpyxl import (  # NOQA: backwards compatibility for plugins using excel_safe
+    SafeWorkbook, remove_invalid_excel_chars as excel_safe,
+)
 
-
-def excel_safe(val):
-    if isinstance(val, Cell):
-        return val
-
-    if not isinstance(val, KNOWN_TYPES):
-        val = str(val)
-
-    if isinstance(val, bytes):
-        val = val.decode("utf-8", errors="ignore")
-
-    if isinstance(val, str):
-        val = re.sub(ILLEGAL_CHARACTERS_RE, '', val)
-
-    return val
+__ = excel_safe  # just so the compatbility import above is "used" and doesn't get removed by linter
 
 
 class BaseExporter:
@@ -228,7 +214,7 @@ class ListExporter(BaseExporter):
         pass
 
     def _render_xlsx(self, form_data, output_file=None):
-        wb = Workbook(write_only=True)
+        wb = SafeWorkbook(write_only=True)
         ws = wb.create_sheet()
         self.prepare_xlsx_sheet(ws)
         try:
@@ -242,7 +228,7 @@ class ListExporter(BaseExporter):
                 total = line.total
                 continue
             ws.append([
-                excel_safe(val) for val in line
+                val for val in line
             ])
             if total:
                 counter += 1
@@ -347,7 +333,7 @@ class MultiSheetListExporter(ListExporter):
             return self.get_filename() + '.csv', 'text/csv', output.getvalue().encode("utf-8")
 
     def _render_xlsx(self, form_data, output_file=None):
-        wb = Workbook(write_only=True)
+        wb = SafeWorkbook(write_only=True)
         n_sheets = len(self.sheets)
         for i_sheet, (s, l) in enumerate(self.sheets):
             ws = wb.create_sheet(str(l))
@@ -361,8 +347,7 @@ class MultiSheetListExporter(ListExporter):
                     total = line.total
                     continue
                 ws.append([
-                    excel_safe(val)
-                    for val in line
+                    val for val in line
                 ])
                 if total:
                     counter += 1

src/pretix/base/forms/questions.py

@@ -692,7 +692,7 @@ class BaseQuestionsForm(forms.Form):
                     label=label, required=required,
                     min_value=q.valid_number_min or Decimal('0.00'),
                     max_value=q.valid_number_max,
-                    help_text=q.help_text,
+                    help_text=help_text,
                     initial=initial.answer if initial else None,
                 )
             elif q.type == Question.TYPE_STRING:

src/pretix/base/services/mail.py

@@ -77,7 +77,7 @@ from pretix.base.signals import email_filter, global_email_filter
 from pretix.celery_app import app
 from pretix.helpers.hierarkey import clean_filename
 from pretix.multidomain.urlreverse import build_absolute_uri
-from pretix.presale.ical import get_private_icals
+from pretix.presale.ical import get_ical
 
 logger = logging.getLogger('pretix.base.mail')
 INVALID_ADDRESS = 'invalid-pretix-mail-address'
@@ -430,7 +430,18 @@ def mail_send_task(self, *args, to: List[str], subject: str, body: str, html: st
                                     }
                                 )
                         if attach_ical:
-                            for i, cal in enumerate(get_private_icals(event, [position] if position else order.positions.all())):
+                            ical_events = set()
+                            if event.has_subevents:
+                                if position:
+                                    ical_events.add(position.subevent)
+                                else:
+                                    for p in order.positions.all():
+                                        ical_events.add(p.subevent)
+                            else:
+                                ical_events.add(order.event)
+
+                            for i, e in enumerate(ical_events):
+                                cal = get_ical([e])
                                 email.attach('event-{}.ics'.format(i), cal.serialize(), 'text/calendar')
 
             email = email_filter.send_chained(event, 'message', message=email, order=order, user=user)

src/pretix/base/services/orders.py

@@ -932,7 +932,7 @@ def _create_order(event: Event, email: str, positions: List[CartPosition], now_d
 
 
 def _order_placed_email(event: Event, order: Order, pprov: BasePaymentProvider, email_template, log_entry: str,
-                        invoice, payment: OrderPayment, is_free=False):
+                        invoice, payment: OrderPayment):
     email_context = get_email_context(event=event, order=order, payment=payment if pprov else None)
     email_subject = _('Your order: %(code)s') % {'code': order.code}
     try:
@@ -941,7 +941,7 @@ def _order_placed_email(event: Event, order: Order, pprov: BasePaymentProvider,
             log_entry,
             invoices=[invoice] if invoice and event.settings.invoice_email_attachment else [],
             attach_tickets=True,
-            attach_ical=event.settings.mail_attach_ical and (not event.settings.mail_attach_ical_paid_only or is_free),
+            attach_ical=event.settings.mail_attach_ical,
             attach_other_files=[a for a in [
                 event.settings.get('mail_attachment_new_order', as_type=str, default='')[len('file://'):]
             ] if a],
@@ -950,7 +950,7 @@ def _order_placed_email(event: Event, order: Order, pprov: BasePaymentProvider,
         logger.exception('Order received email could not be sent')
 
 
-def _order_placed_email_attendee(event: Event, order: Order, position: OrderPosition, email_template, log_entry: str, is_free=False):
+def _order_placed_email_attendee(event: Event, order: Order, position: OrderPosition, email_template, log_entry: str):
     email_context = get_email_context(event=event, order=order, position=position)
     email_subject = _('Your event registration: %(code)s') % {'code': order.code}
 
@@ -961,7 +961,7 @@ def _order_placed_email_attendee(event: Event, order: Order, position: OrderPosi
             invoices=[],
             attach_tickets=True,
             position=position,
-            attach_ical=event.settings.mail_attach_ical and (not event.settings.mail_attach_ical_paid_only or is_free),
+            attach_ical=event.settings.mail_attach_ical,
             attach_other_files=[a for a in [
                 event.settings.get('mail_attachment_new_order', as_type=str, default='')[len('file://'):]
             ] if a],
@@ -1070,13 +1070,11 @@ def _perform_order(event: Event, payment_provider: str, position_ids: List[str],
             email_attendees_template = event.settings.mail_text_order_placed_attendee
 
         if sales_channel in event.settings.mail_sales_channel_placed_paid:
-            _order_placed_email(event, order, pprov, email_template, log_entry, invoice, payment,
-                                is_free=free_order_flow)
+            _order_placed_email(event, order, pprov, email_template, log_entry, invoice, payment)
             if email_attendees:
                 for p in order.positions.all():
                     if p.addon_to_id is None and p.attendee_email and p.attendee_email != order.email:
-                        _order_placed_email_attendee(event, order, p, email_attendees_template, log_entry,
-                                                     is_free=free_order_flow)
+                        _order_placed_email_attendee(event, order, p, email_attendees_template, log_entry)
 
     return order.id
 

src/pretix/base/settings.py

@@ -1573,32 +1573,6 @@ DEFAULTS = {
             help_text=_("If enabled, we will attach an .ics calendar file to order confirmation emails."),
         )
     },
-    'mail_attach_ical_paid_only': {
-        'default': 'False',
-        'type': bool,
-        'form_class': forms.BooleanField,
-        'serializer_class': serializers.BooleanField,
-        'form_kwargs': dict(
-            label=_("Attach calendar files only after order has been paid"),
-            help_text=_("Use this if you e.g. put a private access link into the calendar file to make sure people only "
-                        "receive it after their payment was confirmed."),
-        )
-    },
-    'mail_attach_ical_description': {
-        'default': '',
-        'type': LazyI18nString,
-        'form_class': I18nFormField,
-        'form_kwargs': dict(
-            label=_("Event description"),
-            widget=I18nTextarea,
-            help_text=_(
-                "You can use this to share information with your attendees, such as travel information or the link to a digital event. "
-                "If you keep it empty, we will put a link to the event shop, the admission time, and your organizer name in there. "
-                "We do not allow using placeholders with sensitive person-specific data as calendar entries are often shared with an "
-                "unspecified number of people."
-            ),
-        )
-    },
     'mail_prefix': {
         'default': None,
         'type': str,

src/pretix/base/views/redirect.py

@@ -23,8 +23,8 @@ import urllib.parse
 
 from django.core import signing
 from django.http import HttpResponseBadRequest, HttpResponseRedirect
-from django.shortcuts import render
 from django.urls import reverse
+from django.shortcuts import render
 
 
 def _is_samesite_referer(request):

src/pretix/control/forms/event.py

@@ -872,8 +872,6 @@ class MailSettingsForm(SettingsForm):
         'mail_attach_ical',
         'mail_attach_tickets',
         'mail_attachment_new_order',
-        'mail_attach_ical_paid_only',
-        'mail_attach_ical_description',
     ]
 
     mail_sales_channel_placed_paid = forms.MultipleChoiceField(
@@ -1081,8 +1079,7 @@ class MailSettingsForm(SettingsForm):
         'mail_text_download_reminder_attendee': ['event', 'order', 'position'],
         'mail_text_resend_link': ['event', 'order'],
         'mail_text_waiting_list': ['event', 'waiting_list_entry'],
-        'mail_text_resend_all_links': ['event', 'orders'],
-        'mail_attach_ical_description': ['event', 'event_or_subevent'],
+        'mail_text_resend_all_links': ['event', 'orders']
     }
 
     def _set_field_placeholders(self, fn, base_parameters):

src/pretix/control/templates/pretixcontrol/event/mail.html

@@ -14,6 +14,7 @@
                 <legend>{% trans "General" %}</legend>
                 {% bootstrap_field form.mail_prefix layout="control" %}
                 {% bootstrap_field form.mail_attach_tickets layout="control" %}
+                {% bootstrap_field form.mail_attach_ical layout="control" %}
                 {% url "control:organizer.settings.mail" organizer=request.organizer.slug as org_url %}
                 {% propagated request.event org_url "mail_from" "smtp_use_custom" "smtp_host" "smtp_port" "smtp_username" "smtp_password" "smtp_use_tls" "smtp_use_ssl" %}
                     <div class="form-group">
@@ -56,12 +57,6 @@
                 {% endpropagated %}
                 {% bootstrap_field form.mail_sales_channel_placed_paid layout="control" %}
             </fieldset>
-            <fieldset>
-                <legend>{% trans "Calendar invites" %}</legend>
-                {% bootstrap_field form.mail_attach_ical layout="control" %}
-                {% bootstrap_field form.mail_attach_ical_paid_only layout="control" %}
-                {% bootstrap_field form.mail_attach_ical_description layout="control" %}
-            </fieldset>
             <fieldset>
                 <legend>{% trans "E-mail design" %}</legend>
                 <div class="row">

src/pretix/control/templates/pretixcontrol/items/index.html

@@ -46,7 +46,7 @@
                 {% for c in cat_list %}
                     <tbody data-dnd-url="{% url "control:event.items.reorder" organizer=request.event.organizer.slug event=request.event.slug %}">
                     {% for i in c.list %}
-                        {% if forloop.counter0 == 0 and i.category %}<tr class="sortable-disabled"><th colspan="8" scope="colgroup" class="text-muted">{{ i.category }}</th></tr>{% endif %}
+                        {% if forloop.counter0 == 0 and i.category %}<tr class="sortable-disabled"><th colspan="8" scope="colgroup" class="text-muted">{{ i.category.name }}</th></tr>{% endif %}
                         <tr data-dnd-id="{{ i.id }}" {% if not i.active %}class="row-muted"{% endif %}>
                             <td><strong>
                                 {% if not i.active %}<strike>{% endif %}

src/pretix/control/templates/pretixcontrol/items/quota_delete.html

@@ -13,7 +13,7 @@
                 <ul>
                     {% for item in dependent %}
                         <li>
-                            <a href="{% url "control:event.item" organizer=request.event.organizer.slug event=request.event.slug item=item.pk %}">{{ item }}</a>
+                            <a href="{% url "control:event.item" organizer=request.event.organizer.slug event=request.event.slug item=item.pk %}">{{ item.name }}</a>
                         </li>
                     {% endfor %}
                 </ul>

src/pretix/helpers/safe_openpyxl.py

@@ -0,0 +1,113 @@
+import re
+from inspect import isgenerator
+
+from openpyxl import Workbook
+from openpyxl.cell.cell import (
+    ILLEGAL_CHARACTERS_RE, KNOWN_TYPES, TIME_TYPES, TYPE_FORMULA, TYPE_STRING,
+    Cell,
+)
+from openpyxl.compat import NUMERIC_TYPES
+from openpyxl.utils import column_index_from_string
+from openpyxl.utils.exceptions import ReadOnlyWorkbookException
+from openpyxl.worksheet._write_only import WriteOnlyWorksheet
+from openpyxl.worksheet.worksheet import Worksheet
+
+SAFE_TYPES = NUMERIC_TYPES + TIME_TYPES + (bool, type(None))
+
+
+"""
+This module provides a safer version of openpyxl's `Workbook` class to generate XLSX files from
+user-generated data using `WriteOnlyWorksheet` and `ws.append()`. We commonly use these methods
+to output e.g. order data, which contains data from untrusted sources such as attendee names.
+
+There are mainly two problems this solves:
+
+- It makes sure strings starting with = are treated as text, not as a formula, as openpyxl will
+  otherwise assume, which can be used for remote code execution.
+
+- It removes characters considered invalid by Excel to avoid exporter crashes.
+"""
+
+
+def remove_invalid_excel_chars(val):
+    if isinstance(val, Cell):
+        return val
+
+    if not isinstance(val, KNOWN_TYPES):
+        val = str(val)
+
+    if isinstance(val, bytes):
+        val = val.decode("utf-8", errors="ignore")
+
+    if isinstance(val, str):
+        val = re.sub(ILLEGAL_CHARACTERS_RE, '', val)
+
+    return val
+
+
+def SafeCell(*args, value=None, **kwargs):
+    value = remove_invalid_excel_chars(value)
+    c = Cell(*args, value=value, **kwargs)
+    if c.data_type == TYPE_FORMULA:
+        c.data_type = TYPE_STRING
+    return c
+
+
+class SafeAppendMixin:
+    def append(self, iterable):
+        row_idx = self._current_row + 1
+
+        if isinstance(iterable, (list, tuple, range)) or isgenerator(iterable):
+            for col_idx, content in enumerate(iterable, 1):
+                if isinstance(content, Cell):
+                    # compatible with write-only mode
+                    cell = content
+                    if cell.parent and cell.parent != self:
+                        raise ValueError("Cells cannot be copied from other worksheets")
+                    cell.parent = self
+                    cell.column = col_idx
+                    cell.row = row_idx
+                else:
+                    cell = SafeCell(self, row=row_idx, column=col_idx, value=remove_invalid_excel_chars(content))
+                self._cells[(row_idx, col_idx)] = cell
+
+        elif isinstance(iterable, dict):
+            for col_idx, content in iterable.items():
+                if isinstance(col_idx, str):
+                    col_idx = column_index_from_string(col_idx)
+                cell = SafeCell(self, row=row_idx, column=col_idx, value=content)
+                self._cells[(row_idx, col_idx)] = cell
+
+        else:
+            self._invalid_row(iterable)
+
+        self._current_row = row_idx
+
+
+class SafeWriteOnlyWorksheet(SafeAppendMixin, WriteOnlyWorksheet):
+    pass
+
+
+class SafeWorksheet(SafeAppendMixin, Worksheet):
+    pass
+
+
+class SafeWorkbook(Workbook):
+    def __init__(self, *args, **kwargs):
+        super().__init__(*args, **kwargs)
+        if self._sheets:
+            # monkeypatch existing sheets
+            for s in self._sheets:
+                s.append = SafeAppendMixin.append
+
+    def create_sheet(self, title=None, index=None):
+        if self.read_only:
+            raise ReadOnlyWorkbookException('Cannot create new sheet in a read-only workbook')
+
+        if self.write_only:
+            new_ws = SafeWriteOnlyWorksheet(parent=self, title=title)
+        else:
+            new_ws = SafeWorksheet(parent=self, title=title)
+
+        self._add_sheet(sheet=new_ws, index=index)
+        return new_ws

src/pretix/presale/forms/checkout.py

@@ -47,6 +47,7 @@ from pretix.base.forms.questions import (
     BaseInvoiceAddressForm, BaseQuestionsForm, WrappedPhoneNumberPrefixWidget,
     guess_phone_prefix,
 )
+from pretix.base.templatetags.rich_text import rich_text
 from pretix.base.validators import EmailBanlistValidator
 from pretix.presale.signals import contact_form_fields
 
@@ -82,7 +83,7 @@ class ContactForm(forms.Form):
             self.fields['phone'] = PhoneNumberField(
                 label=_('Phone number'),
                 required=self.event.settings.order_phone_required,
-                help_text=self.event.settings.checkout_phone_helptext,
+                help_text=rich_text(self.event.settings.checkout_phone_helptext),
                 widget=WrappedPhoneNumberPrefixWidget()
             )
 
@@ -91,7 +92,7 @@ class ContactForm(forms.Form):
             # is an autofocus field. Who would have thought… See e.g. here:
             # https://floatboxjs.com/forum/topic.php?post=8440&usebb_sid=2e116486a9ec6b7070e045aea8cded5b#post8440
             self.fields['email'].widget.attrs['autofocus'] = 'autofocus'
-        self.fields['email'].help_text = self.event.settings.checkout_email_helptext
+        self.fields['email'].help_text = rich_text(self.event.settings.checkout_email_helptext)
 
         responses = contact_form_fields.send(self.event, request=self.request)
         for r, response in responses:

src/pretix/presale/forms/waitinglist.py

@@ -28,6 +28,7 @@ from pretix.base.forms.questions import (
     NamePartsFormField, WrappedPhoneNumberPrefixWidget, guess_phone_prefix,
 )
 from pretix.base.models import Quota, WaitingListEntry
+from pretix.base.templatetags.rich_text import rich_text
 from pretix.presale.views.event import get_grouped_items
 
 
@@ -99,7 +100,7 @@ class WaitingListForm(forms.ModelForm):
             self.fields['phone'] = PhoneNumberField(
                 label=_("Phone number"),
                 required=event.settings.waiting_list_phones_required,
-                help_text=event.settings.waiting_list_phones_explanation_text,
+                help_text=rich_text(event.settings.waiting_list_phones_explanation_text),
                 widget=WrappedPhoneNumberPrefixWidget()
             )
         else:

src/pretix/presale/ical.py

@@ -28,16 +28,11 @@ from django.conf import settings
 from django.utils.formats import date_format
 from django.utils.translation import gettext as _
 
-from pretix.base.email import get_email_context
 from pretix.base.models import Event
 from pretix.multidomain.urlreverse import build_absolute_uri
 
 
-def get_public_ical(events):
-    """
-    Return an ical feed for a sequence of events or subevents. The calendar files will only include public
-    information.
-    """
+def get_ical(events):
     cal = vobject.iCalendar()
     cal.add('prodid').value = '-//pretix//{}//'.format(settings.PRETIX_INSTANCE_NAME.replace(" ", "_"))
     creation_time = datetime.datetime.now(pytz.utc)
@@ -88,91 +83,3 @@ def get_public_ical(events):
 
         vevent.add('description').value = '\n'.join(descr)
     return cal
-
-
-def get_private_icals(event, positions):
-    """
-    Return a list of ical objects based on a sequence of positions.
-
-    Unlike get_public_ical, this will
-
-    - Generate multiple ical files instead of one (but with deduplication applied)
-    - Respect the mail_attach_ical_description setting
-
-    It is private in the sense that mail_attach_ical_description may contain content not suited for
-    public display.
-
-    We however intentionally do not allow using placeholders based on the order and position
-    specifically. This is for two reasons:
-
-    - In reality, many people will add their invite to their calendar which is shared with a larger
-      team. People are probably not aware that they're sharing sensitive information such as their
-      secret ticket link with everyone they share their calendar with.
-
-    - It would be pretty hard to implement it in a way that doesn't require us to use distinct
-      settings fields for emails to customers and to attendees, which feels like an overcomplication.
-    """
-
-    from pretix.base.services.mail import TolerantDict
-
-    tz = pytz.timezone(event.settings.timezone)
-
-    creation_time = datetime.datetime.now(pytz.utc)
-    calobjects = []
-
-    evs = set(p.subevent or event for p in positions)
-    for ev in evs:
-        if isinstance(ev, Event):
-            url = build_absolute_uri(event, 'presale:event.index')
-        else:
-            url = build_absolute_uri(event, 'presale:event.index', {
-                'subevent': ev.pk
-            })
-
-        if event.settings.mail_attach_ical_description:
-            ctx = get_email_context(event=event, event_or_subevent=ev)
-            description = str(event.settings.mail_attach_ical_description).format_map(TolerantDict(ctx))
-        else:
-            # Default description
-            descr = []
-            descr.append(_('Tickets: {url}').format(url=url))
-            if ev.date_admission:
-                descr.append(str(_('Admission: {datetime}')).format(
-                    datetime=date_format(ev.date_admission.astimezone(tz), 'SHORT_DATETIME_FORMAT')
-                ))
-
-            descr.append(_('Organizer: {organizer}').format(organizer=event.organizer.name))
-            description = '\n'.join(descr)
-
-        cal = vobject.iCalendar()
-        cal.add('prodid').value = '-//pretix//{}//'.format(settings.PRETIX_INSTANCE_NAME.replace(" ", "_"))
-
-        vevent = cal.add('vevent')
-        vevent.add('summary').value = str(ev.name)
-        vevent.add('description').value = description
-        vevent.add('dtstamp').value = creation_time
-        if ev.location:
-            vevent.add('location').value = str(ev.location)
-
-        vevent.add('uid').value = 'pretix-{}-{}-{}@{}'.format(
-            event.organizer.slug,
-            event.organizer.slug, event.slug,
-            ev.pk if not isinstance(ev, Event) else '0',
-            urlparse(url).netloc
-        )
-
-        if event.settings.show_times:
-            vevent.add('dtstart').value = ev.date_from.astimezone(tz)
-        else:
-            vevent.add('dtstart').value = ev.date_from.astimezone(tz).date()
-
-        if event.settings.show_date_to and ev.date_to:
-            if event.settings.show_times:
-                vevent.add('dtend').value = ev.date_to.astimezone(tz)
-            else:
-                # with full-day events date_to in pretix is included (e.g. last day)
-                # whereas dtend in vcalendar is non-inclusive => add one day for export
-                vevent.add('dtend').value = ev.date_to.astimezone(tz).date() + datetime.timedelta(days=1)
-
-        calobjects.append(cal)
-    return calobjects

src/pretix/presale/views/event.py

@@ -70,7 +70,7 @@ from pretix.base.models.items import (
 from pretix.base.services.quotas import QuotaAvailability
 from pretix.helpers.compat import date_fromisocalendar
 from pretix.multidomain.urlreverse import eventreverse
-from pretix.presale.ical import get_public_ical
+from pretix.presale.ical import get_ical
 from pretix.presale.signals import item_description
 from pretix.presale.views.organizer import (
     EventListMixin, add_subevents_for_days, days_for_template,
@@ -719,7 +719,7 @@ class EventIcalDownload(EventViewMixin, View):
                 raise Http404(pgettext_lazy('subevent', 'Unknown date selected.'))
 
         event = self.request.event
-        cal = get_public_ical([subevent or event])
+        cal = get_ical([subevent or event])
 
         resp = HttpResponse(cal.serialize(), content_type='text/calendar')
         resp['Content-Disposition'] = 'attachment; filename="{}-{}-{}.ics"'.format(

src/pretix/presale/views/organizer.py

@@ -67,7 +67,7 @@ from pretix.helpers.formats.en.formats import (
     SHORT_MONTH_DAY_FORMAT, WEEK_FORMAT,
 )
 from pretix.multidomain.urlreverse import eventreverse
-from pretix.presale.ical import get_public_ical
+from pretix.presale.ical import get_ical
 from pretix.presale.views import OrganizerViewMixin
 
 
@@ -1159,9 +1159,9 @@ class OrganizerIcalDownload(OrganizerViewMixin, View):
 
         if 'locale' in request.GET and request.GET.get('locale') in dict(settings.LANGUAGES):
             with language(request.GET.get('locale'), self.request.organizer.settings.region):
-                cal = get_public_ical(events)
+                cal = get_ical(events)
         else:
-            cal = get_public_ical(events)
+            cal = get_ical(events)
 
         resp = HttpResponse(cal.serialize(), content_type='text/calendar')
         resp['Content-Disposition'] = 'attachment; filename="{}.ics"'.format(

src/pretix/settings.py

@@ -97,8 +97,6 @@ else:
             f.write(SECRET_KEY)
 
 # Adjustable settings
-LANGUAGE_CODE = config.get('locale', 'default', fallback='en')
-TIME_ZONE = config.get('locale', 'timezone', fallback='UTC')
 
 debug_fallback = "runserver" in sys.argv
 DEBUG = config.getboolean('django', 'debug', fallback=debug_fallback)
@@ -132,7 +130,6 @@ DATABASES = {
         'HOST': config.get('database', 'host', fallback=''),
         'PORT': config.get('database', 'port', fallback=''),
         'CONN_MAX_AGE': 0 if db_backend == 'sqlite3' else 120,
-        'TIME_ZONE': TIME_ZONE,
         'OPTIONS': db_options,
         'TEST': {
             'CHARSET': 'utf8mb4',
@@ -151,7 +148,6 @@ if config.has_section('replica'):
         'HOST': config.get('replica', 'host', fallback=DATABASES['default']['HOST']),
         'PORT': config.get('replica', 'port', fallback=DATABASES['default']['PORT']),
         'CONN_MAX_AGE': 0 if db_backend == 'sqlite3' else 120,
-        'TIME_ZONE': TIME_ZONE,
         'OPTIONS': db_options,
         'TEST': {
             'CHARSET': 'utf8mb4',
@@ -214,6 +210,9 @@ CURRENCY_PLACES = {
 
 ALLOWED_HOSTS = ['*']
 
+LANGUAGE_CODE = config.get('locale', 'default', fallback='en')
+TIME_ZONE = config.get('locale', 'timezone', fallback='UTC')
+
 MAIL_FROM = SERVER_EMAIL = DEFAULT_FROM_EMAIL = config.get('mail', 'from', fallback='pretix@localhost')
 MAIL_FROM_NOTIFICATIONS = config.get('mail', 'from_notifications', fallback=MAIL_FROM)
 MAIL_FROM_ORGANIZERS = config.get('mail', 'from_organizers', fallback=MAIL_FROM)

src/pretix/static/pretixpresale/js/ui/main.js

@@ -219,7 +219,10 @@ $(function () {
             // multi-input fields have a role=group with aria-labelledby 
             var label = this.hasAttribute("aria-labelledby") ? $("#" + this.getAttribute("aria-labelledby")) : $("[for="+target.attr("id")+"]");
 
-            content.append("<li><a href='#" + target.attr("id") + "'>" + label.get(0).childNodes[0].nodeValue + "</a>: "+desc.text()+"</li>");
+            var $li = $("<li>");
+            $li.text(": " + desc.text())
+            $li.prepend($("<a>").attr("href", "#" + target.attr("id")).text(label.get(0).childNodes[0].nodeValue))
+            content.append($li);
         });
         $(this).append(content);
     });

src/setup.py

@@ -172,7 +172,7 @@ setup(
         'Django==3.2.*',
         'django-bootstrap3==15.0.*',
         'django-compressor==2.4.*',
-        'django-countries>=7.2',
+        'django-countries==7.2.*',
         'django-filter==21.1',
         'django-formset-js-improved==0.5.0.2',
         'django-formtools==2.3',

src/tests/helpers/test_safe_openpyxl.py

@@ -0,0 +1,38 @@
+#
+# This file is part of pretix (Community Edition).
+#
+# Copyright (C) 2014-2020 Raphael Michel and contributors
+# Copyright (C) 2020-2021 rami.io GmbH and contributors
+#
+# This program is free software: you can redistribute it and/or modify it under the terms of the GNU Affero General
+# Public License as published by the Free Software Foundation in version 3 of the License.
+#
+# ADDITIONAL TERMS APPLY: Pursuant to Section 7 of the GNU Affero General Public License, additional terms are
+# applicable granting you additional permissions and placing additional restrictions on your usage of this software.
+# Please refer to the pretix LICENSE file to obtain the full terms applicable to this work. If you did not receive
+# this file, see <https://pretix.eu/about/en/license>.
+#
+# This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied
+# warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU Affero General Public License for more
+# details.
+#
+# You should have received a copy of the GNU Affero General Public License along with this program.  If not, see
+# <https://www.gnu.org/licenses/>.
+#
+from openpyxl.cell.cell import TYPE_STRING
+
+from pretix.helpers.safe_openpyxl import SafeWorkbook
+
+
+def test_nullbyte_removed():
+    wb = SafeWorkbook()
+    ws = wb.create_sheet()
+    ws.append(["foo\u0000bar"])
+    assert ws.cell(1, 1).value == "foobar"
+
+
+def test_no_formulas():
+    wb = SafeWorkbook()
+    ws = wb.create_sheet()
+    ws.append(["=1+1"])
+    assert ws.cell(1, 1).data_type == TYPE_STRING