Interactive, user-friendly database explorer.

There is a live demo site available here https://data-browser-demo.herokuapp.com/data-browser/.

Because it's hosted on Heroku free tier it might take a while to respond to the first page load.

The Django project is a small e-commerce site selling microservices.

Source: https://github.com/tolomea/data-browser-demo.

Admin: https://data-browser-demo.herokuapp.com/admin/.

• Zero config, if it's in the admin it's in the browser.
• Select fields (including calculated fields), aggregate, filter, sort and pivot.
• Automatically follow OneToOneFields and ForeignKeys.
• Respects per user admin permissions.
• Share views simply by sharing URLs.
• Save views and optionally make them available to services like Google sheets.
• Download views as CSV or JSON.

The Data Browser is currently tested on:

• Django 3.2 - 5.0
• Python 3.8 - 3.12
• MySQL, PostgreSQL, SQLite

We highly recommend and only officially support the latest patch release of each Python and Django series.

1. Run pip install django-data-browser.
2. Add "data_browser" to installed_apps.
3. Add path("data-browser/", include("data_browser.urls")) to your urls.
4. Run python manage.py migrate.
5. If you have queryset annotations in your admin or are interested in exposing calculated values see the Calculated and Annotated fields section.

Most of the Django views in the Data Browser can only be accessed by Django "staff members". These views support general querying of the database, checked against the admin permissions of the logged in user.

The only exception to this is "Public Saved Views" these are views which have been saved and marked as public. They can be accessed by anyone without needing a login but they can only be used to access a query that has been saved and made public and they have long random URL's.

You can use the admin permission data_browser | view | Can make a saved view publically available to restrict who can make views public. To be public the view must be marked as public and owned by someone who has the permission.

Additionally the entire public views system is gated by the Django settings value DATA_BROWSER_ALLOW_PUBLIC.

The frontend code has builtin Sentry support, it is disabled by default. To enable it set the Django settings value DATA_BROWSER_FE_DSN, for example to set it to the Data Browser project Sentry use:

DATA_BROWSER_FE_DSN = "https://af64f22b81994a0e93b82a32add8cb2b@o390136.ingest.sentry.io/5231151"

The home page URL of the Data Browser is given by reverse("data_browser:home").

Additionally if you are using data_browser.helpers.AdminMixin then in Admin list views the URL of the Data Browser page for the same model is available as the template context variable ddb_url.

One convenient way of utilizing this is to create the file templates/admin/change_list_object_tools.html and populate it with:

{% extends "admin/change_list_object_tools.html" %}
{% block object-tools-items %}
    {{ block.super }}
    {% if ddb_url %}
        <li><a href="{{ ddb_url }}" class="viewlink">Data Browser</a></li>
    {% endif %}
{% endblock %}

This will place a "Data Browser" button on the list view of every admin that inherits from the mixin. Note: to do this at the top level, the app you put the template in must be before contrib.admin in INSTALLED_APPS.

By default the Data Browser has access to all models and fields that the current user can see anywhere in the Admin site. However if necessary this can be tweaked using the following class level properties and functions on ModelAdmins and Inlines.

These can also be set via the settings entry DATA_BROWSER_ADMIN_OPTIONS, this is useful if you wish to change an option on a third party admin. The format for this is {'my_package.MyAdmin': {'option': value}}, for example to remove Django Q's Task and Fail admins entirely you would use:

DATA_BROWSER_ADMIN_OPTIONS = {
    'django_q.admin.TaskAdmin': {'ignore': True},
    'django_q.admin.FailAdmin': {'ignore': True},
}

Finally, per the below sections, calculated fields and actions can be hidden by setting the ddb_hide attribute and annotated fields are always visible unless explicitly hidden.

Calculated fields are fields on the ModelAdmin whose value comes from a function on the ModelAdmin or a function or property on the Model itself, as described at the bottom of the Django admin list display docs.

Being arbitrary Python code calculated fields are opaque to the Data Browser. It can fetch their values but can't sort or filter etc on them. For pivoting they are treated as equivalent to the pk on the same model.

Additionally calculated fields can be hidden from the Data Browser by setting the attribute ddb_hide to True. The data_browser.helpers.attributes decorator can make this a little tidier.

@attributes(ddb_hide=True)
def my_calculated_field(self, obj):
    return ...

The Data Browser has additional support for annotated fields. Normally you would expose these as calculated fields. The module data_browser.helpers contains helpers which will make exposing annotated fields simpler, more performant and expose them to the Data Browser so it can do arbitrary manipulation with them.

Exposing an annotated field in this way requires two changes.

1. Mix data_browser.helpers.AdminMixin into your ModelAdmin.
2. Add a function decorated with data_browser.helpers.annotation that takes and updates a queryset.

from data_browser.helpers import AdminMixin, annotation

@admin.register(MyModel)
class MyAdmin(AdminMixin, ModelAdmin):
    fields = ["my_field"]

    @annotation
    def my_field(self, request, qs):
        return qs.annotate(my_field=Cast(..., output_field=IntegerField()))

WARNING: annotated aggregations will produce misleading results when further aggregated in the Data Browser.

It is important that the decorated annotation function name and the annotated queryset field name match.

Sometimes it is necessary for the top level of the annotation to have output_field set so the Data Browser can tell what type of data it will produce. When this is necessary you will get an error to that effect.

The helpers will automatically deal with the admin_order_field and boolean properties and readonly_fields, reducing the boiler plate involved in using annotations in the admin.

Additionally the annotation will only be applied to the list view when it's mentioned in list_display this allows you to use annotations extensively on your detail views without hurting the performance of your list views.

And finally even if not mentioned in fields, fieldsets or list_display, the annotation will still be visible in the Data Browser unless it is explicitly mentioned in ddb_hide_fields.

The Data Browser does it's fetching in two stages.

First it does a single DB query to get the majority of the data. To construct the queryset for this it will call get_queryset() on the ModelAdmin of the current Model. It uses .values() to fetch only the data it needs from the database and it will inline all referenced models to ensure it doesn't do multiple queries.

At this stage annotated fields on related models are attached with subquery annotations, the data browser will call get_queryset() on the relevant ModelAdmins in order to generate these subquery annotations.

Secondly for any calculated fields it will then fetch the complete objects that are needed for those calculated fields. To construct the querysets for these it will call get_queryset() on their associated ModelAdmins. These calls are aggregated so it will only make one per model.

As a simple example, if you did a query against the Book model for the fields:

• book.name
• book.author.name
• book.author.age
• book.author.number_of_books
• book.publisher.name

Where the author.age is actually a property on the Author model, and author.number_of_books is an @annotation on the Author ModelAdmin, then it would do something like the following two queries:

BookAdmin.get_queryset().annotate(
    author__number_of_books=Subquery(
        AuthorAdmin.get_queryset()
        .filter(pk=OuterRef("author__id"))
        .values("number_of_books")[:1]
    )
).values(
    "name",
    "author__name",
    "author__id",
    "author__number_of_books",
    "publisher__name",
)
AuthorAdmin.get_queryset().in_bulk(pks=...)

Where the pks passed to in_bulk in the second query came from author__id in the first.

You can view an approximation of the main queryset by changing the .html in the URL to .qs. In a similar manner .sql, .explain and .analyze are also available.

When the Data Browser calls the admin get_queryset() functions it will put some context in request.data_browser. This allows you to test to see if the Data Browser is making the call as follows:

if hasattr(request, "data_browser"):
    # Data Browser specific customization

This is particularly useful if you want to route the Data Browser to a DB replica for a particular model (n.b. if you want to do this for all models see QuerySet.using() below.).

The context also includes a fields member that lists all the fields the Data Browser plans to access. You can use this to do conditional prefetching or annotating to support those fields like this:

if (
    not hasattr(request, "data_browser")
    or "my_field" in request.data_browser["fields"]
):
    # do prefetching and annotating associated with my_field

The AdminMixin described in the Calculated and Annotated fields section is doing this internally for @annotation fields.

The setting DATA_BROWSER_USING_DB can be used to direct Data Browser initiated database queries to a replica. Underneath the value of this is passed into QuerySet.using().

The Data Browser also calls get_fieldsets to find out what fields the current user can access.

As with get_queryset the Data Browser will set request.data_browser when calling get_fieldsets and you can test this to detect it and make Data Browser specific customizations.

The Django User Admin has code to change the fieldsets when adding a new user. To compensate for this, when calling get_fieldsets on a subclass of django.contrib.auth.admin.UserAdmin the Data Browser will pass a newly constructed instance of the relevant model. This behavior can be disabled by setting settings.DATA_BROWSER_AUTH_USER_COMPAT to False.

Django's Admin actions are exposed by right clicking on ID (or other appropriate pk field) column headers.

Due to the way these are implemented in Django there are some additional technical considerations.

The actions are posted to the Admin changelist URL. Once this post happens the Data Browser is no longer involved and so can't set request.data_browser like it normally would. Instead it will set the post argument data_browser.

When the Data Browser triggers actions default Admin filtering is applied. If you have Admin filters that hide rows by default then actions triggered from the Data Browser will not be able to access those rows. To work around this you can specify get_ddb_action_url to override the URL the actions are posted to. By default it returns the changelist URL so you can append any arguments needed to set filters to not filter.

You can override the data_browser/index.html template per Django’s template overriding docs, and replace the extrahead block. (Ensure "data_browser" is after your app in INSTALLED_APPS.)

This will let you inject custom CSS and stylesheets.

However note that because of how the normal CSS is injected any custom CSS will be before the normal CSS so you will need to use more specific selectors or !important.

It is possible to register additional functions and aggregations with the Data Browser, including custom ones.

Functions and Aggregations are attached to the Data Browsers core types, these are in data_browser.types.

This functionality is provisional and not subject to normal backward compatible guarantees.

The function registry is in data_browser.orm_functions.TYPE_FUNCTIONS this has type dict[BaseType, dict[str, data_browser.orm_functions.Func]]. It is safe to insert new entries into this at runtime.

For example to add the MD5 function to string fields you could do the following:

from django.db.models.functions import MD5
from data_browser.types import StringType
from data_browser.orm_functions import Func, TYPE_FUNCTIONS

TYPE_FUNCTIONS[StringType]["md5"] = Func(MD5, StringType)

This functionality is provisional and not subject to normal backward compatible guarantees.

The aggregate registry is in data_browser.orm_aggregates.TYPE_AGGREGATES this has type dict[BaseType, dict[str, data_browser.orm_aggregates.Agg]]. It is safe to insert new entries into this at runtime.

For example to add a count that does not apply distinct to number fields you could do the following:

from django.db.models import Count
from data_browser.types import NumberType
from data_browser.orm_aggregates import Agg, TYPE_AGGREGATES

TYPE_AGGREGATES[NumberType]["full_count"] = Agg(
    lambda x: Count(x, distinct=False), NumberType
)

For a larger example imagine you wanted to use Postgres's percentile_cont functionality to add a p95 aggregate to duration fields, perhaps for some kind of application performance monitoring usecase.

First we need to explain percentile_cont to Django.

from django.db.models import Aggregate

class Percentile(Aggregate):
    arity = 1
    function = "percentile_cont"
    template = "%(function)s(%(percentile)s) WITHIN GROUP (ORDER BY %(expressions)s)"
    name = "Percentile"

    def __init__(self, percentile, expressions, **extra):
        super().__init__(expressions, percentile=percentile, **extra)

Then we need to tell the Data Browser we want p95 on duration fields.

from data_browser.orm_aggregates import TYPE_AGGREGATES, Agg
from data_browser.types import DurationType

TYPE_AGGREGATES[DurationType]["p95"] = Agg(
    lambda x: Percentile(0.95, x), DurationType
)

You can create and use a custom admin.AdminSite (see https://docs.djangoproject.com/en/4.2/ref/contrib/admin/).

To do so, in your settings.py, add:

from django.contrib import admin

class BrowserAdminSite(admin.AdminSite):
    pass

DATA_BROWSER_ADMIN_SITE = BrowserAdminSite(name='data_browser')

Then, in any admin.py, register the models as usually but using DATA_BROWSER_ADMIN_SITE.

For instance in myapp/admin.py:

from django.contrib import admin
from django.conf import settings
from myapp.models import MyAdminModel, MyBrowsableModel

# register in admin only
admin.register(MyAdminModel)

# register in data browser only
settings.DATA_BROWSER_ADMIN_SITE.register(MyBrowsableModel)

The Data Browser uses the standard Major.Minor.Patch version numbering scheme.

Patch versions may include bug fixes and minor features.

Minor versions are for significant new features.

Major versions are for major features, significant changes to existing functionality and breaking changes.

Patch and Minor versions should never contain breaking changes and should always be backward compatible. A breaking change is a change that makes backward incompatible changes to one or more of the following:

• The query URL format.
• The json, csv etc data formats, this does not include the Data Browsers internal API's, only the data export formats.
• The format of the request.data_browser passed to get_fieldsets and get_queryset.
• Existing saved views.
• The URL's of public saved views.

For alpha and beta releases absolutely anything may change / break.