DjangoCon US 2023
Day 1
Orientation and Welcome
“I am going to say some words here I don’t fully understand” – @lacey
Contributing to Django or how I learned to stop worrying and just try to fix an ORM Bug
- Your Web Framework Needs You! – Carlton Gibson
- Ticket Strategy: Something old and straightforward
- Steps:
- Replicate the bug
- Read some docs
- Write some code
- Test the code
- Write down (publicly) what you learned
HTML-ivating your Django web app’s experience with HTMX, AlpineJS, and streaming HTML
SPA’s are popular because they can provide dynamic user experiences with no reloading. This comes at the cost of larger file sizes making pages take longer to load, and higher complexity making the codebase harder to maintain.
We can achieve the same dynamic experiences with tools like HTMX and AlpineJS, while also reducing load times and complexity.
✅ These tools provide the features an elevated user experiences (normally via SPAs):
- (HTMX) Remove whole-page refreshes for every interaction
- (HTMX) Use small payloads from the server to update the interface
- (AlpineJS) Update HTML as a result of changes in data
- (AlpineJS) Empower rich on-page interactions
- (HTMX/AlpineJS) Be fast
✅ These tools provide faster pages loads by literally being small:
- Around 30KB
✅ These tools provide easy to maintain HTML attributes control behavior:
- Locality
- Remove your JS build system
- Write mostly Python and HTML
- Easier maintenance
- Faster iteration cycles
Lightning Talks - Day 1
- April 8, 2024 Total Solar Eclipse
- Last in the continental US until 2044/2045
- By glasses/filters early (ISO 12312-2 certified)
- Django Stickers! 🦄
Using database triggers to reliably track model history
django-pghistory tracks changes to your Django models using Postgres triggers, providing:
- Reliable history tracking everywhere with no changes to your application code.
- Structured history models that mirror the fields of your models.
- Grouping of history with additional context attached, such as the logged-in user.
import pghistory
@pghistory.track()
class TrackedModel(models.Model):
int_field = models.IntegerField()
text_field = models.TextField()
Beyond the Basics of Migrations
- DON’T reinvent the wheel: if Django’s autogenerated migrations work, use that
- DO rename custom migration files for clarity
- DO always use the reverse parameter when using RunPython or RunSQL
- DO make sure your coworkers know if you customized a migration so they can review it properly
- DON’T use migrations for repeat actions
- Efficiency matters on large tables (especially with CI/CD)
Custom Model Managers and QuerySets: Graduating from Django Beginner to ORM Master
- What are Managers? An interface through which database query operations are provided to Django models.
- Model methods = row level operations
- Manager methods = table level operations
- How do you define Managers?
objects = CustomManager() # basic usage objects = CustomQuerySet.as_manager() # bread and butter usage objects = Custommanager.from_queryset(CustomerQuerySet()) # advanced usage
- Manager or QuerySet?
- QuerySets = Chainable, Filters, Annotations/Aggregations, Reading
- Manager = Not Chainable, Creating, Updating, Deleting
- Manager or QuerySet?
- Why use custom Managers?
- Readability
- Encapsulation
- Testing!!!
- Built-in service layer
- Gateway to learning the ORM
- Common Patterns
.for_CONTEXT(ctx)
- Manager or QuerySet
- Filter based on some context
- Ex:
.for_user(user)
.is_CONDITION(cond?) .not_CONDITION(cond?) .exclude_CONDITION(cond?)
- Manager or QuerySet
- Filter based on some condition
- Ex:
.is_leasable() .is_leased_for_year(year)
.create_MODEL() .create_MODEL_for_CONTEXT(ctx)
- Manager
- Create a model with side effects
- Ex:
.create_user_for_club(club)
.within_RANGE()
- Manager or QuerySet
- Filter based on numerical/date range
- Ex:
.within_start_date(days)
.greater_than_VALUE() .less_than_VALUE()
- Manager or QuerySet
- Filter based on numerical/date value
- Ex:
greater_than_days(days)
.set_FIELD() .toggle_FIELD()
- Manager or QuerySet
- Update a field across entire QuerySet
- Ex:
.set_active() .toggle_status()
.with_ANNOTATION()
- Manager or QuerySet
- Add additional info not available on Model
- Ex:
.with_adjusted_income()
Passkeys on Django
See django-passkeys.
Building Powerful APIs with Django, Django Rest Framework, and OpenAPI
- Status codes
- 1xx: Informational —> in progress
- 2xx: Successful —> done
- 3xx: Redirection —> naahh, ain’t nobody got time for that.
- 4xx: Client error —> its definitely you, not me
- 5xx: Server error —> okay, its me not you
- API first approach
- Separation of concerns
- Flexibility and scalability
- Collaboration and parallel development
- Clear and consistent interfaces
- Documentation and contract-driven development
- Future-proofing
- Integration and Third-party support
- Cache all the things!
Hosting and DevOps for Django
- Goals
- (Hosting) Setup is resilient to individual server failures
- (Hosting) Resources can be scaled in response to load
- (Deployment) Code can be tested before being deployed to production
- (Deployment) Deployments are automated
- (Data/Secrets) Data and secrets are secured and backed-up
- (Monitoring) Errors are easy to notice and diagnose
- Hosting
- What are the options?
Bare servers (EC2)- Platform as a Service (EB)
- Managed Container Service (ECS)
- Kubernetes (EKS)
- Why these three?
- These technologies work one level up from servers/containers
- The unit of work is an instance of an application
- Dead servers can be automatically replaced
- Resources can be scaled manually or automatically
- Which one to pick?
- Platform as a Service is a sensible default and a great starting place
- Managed Container Services make sense if you have many services or heavy non-web workloads
- Kubernetes is for multi-cloud or on-prem deployment. Kubernetes can run all your infrastructure, but then you must reinvent a lot of things (esp. load balancers, databases, and block storage)
- What about inbound traffic?
- Load Balancers (ALB)
- Allow multiple application servers without single point of failure
- Allows autoscaling servers (on CPU usage or custom metric)
- Protects from some kinds of DDOS (adding Nginx/Apache can help)
- Can share one load balancer between multiple environments
- Bonus: handles HTTPS and HTTP redirects
- Load Balancers (ALB)
- What about static files?
- Okay: unicorn + whitenoise
- Better: Reverse Proxy (Apache or Nginx)
- Best: CDN (CloudFront or CloudFlair)
- What about background tasks?
- AWS Elastic Beanstalk Workers (weak at on-demand)
- AWS Lambda + SOS (15-minute limit)
- Celery (complex setup)
- What are the options?
- Deployment
- Steps
- Create new running copy of application (code, dependencies, static assets)
- Do peripheral infrastructure changes (migrate, hard part) See Django Migrations: Pitfalls and Solutions
- Put new running copy of application into rotation
- Remove old running copies
- Environments
- Dev Code & Fake Data: Local & Staging
- Dev Code & Real Data: Prod-Copy
- Prod Code & Fake Data: Demo/Training
- Prod Code & Real Data: Production
- Use infrastructure-as-code: Terraform
- CI/CD: GitHub Actions
- System Updates: Covered by PaaS. Don’t forget Docker.
- Steps
- Data/Secrets
- Managed Databases
- Database and OS version upgrades
- Automated backups and managed backup restoration
- Storage autoscaling
- Encryption at rest
- Managed read replicas and failover for prod. Environments (“multi-az”)
- Reduces data loss from 24 hours, to seconds
- Static Files (S3)
- Turn on encryption at rest
- Turn on versioning
- Use django-storages
- Secrets
- Use a secrets manager
- Use django-environ
- Managed Databases
- Monitoring
- Sentry (error monitoring)
- UptimeRobot (uptime monitoring tool)
- DataDog (profiling tool plus error monitoring)
- NewRelic (profiling tool)
- CloudWatch (AWS-native infrastructure monitoring tool)
✨ Modern editing experience for your Django models with Wagtail 🐦
A typical Wagtail project would have your user-facing website structure be defined by the tree structure of the Page model. However, if you have an existing Django project with your own defined Django views, and you don’t want to use Wagtail’s Page model, you can still get plenty of benefits from Wagtail [by integrating the various model mixins into your existing models]!
Day 2
Postgres Performance: From Slow to Pro
- Use database cache (shared buffer)
- Check cache hit ratio (you want ~98%) to know if you need to increase this
- Upgrading versions can increase speed
- PGTune is a good tool for recommended settings!
- Performance tips
- Stop run away queries, set a statement timeout
- Set up
pg_stat_statements
and know your slowest queries. If they’re important, fix them - Add indexes (but not too many) for your longest running and/or your most frequent queries
- Check your cache hit ratio
- Tune memory or add memory as needed
- make sure your connections have enough memory
- make sure you have enough connections, look at
pgbouncer
Empathetic testing: Developing with compassion and humility
- Write tests with a focus on refactorability, outcomes, and uses…rather than defending implementation choices
- Does this test enable, or prevent future changes?
- Seek alternatives to patch/mock (Ex:
pip install responses freezegun
) - This is an “act of kindness” for future developers and creates an enjoyable work experience
Lightning Talks - Day 2
- BUILDING THINGS WHEN YOU HAVE A CLUTTERED MIND
- CONCENTRATION, EVEN IN AN “IDEAL” ENVIRONMENT
- It might be entirely different for you. Experiment.
- Self-help & productivity bloggers do not have all the answers for what works for YOU
- I AM NOT ACCOMPLISHING ENOUGH - OTHERS ARE *AHEAD* OF ME IN LIFE
- You are more than your level of productivity
- Life is not a race with a specified timeline
- SHAME ABOUT HALF-FINISHED OR ABANDONED PROJECTS
- Forgive yourself
- Life isn’t about the # of things you can accomplish
- You are more than your level of productivity
- Not every project needs to be finished
- The journey and experience are valuable
- And, you can almost always go back to that idea or project in the future…
- Take notes. If you don’t write it down, it is not real.
- CONCENTRATION, EVEN IN AN “IDEAL” ENVIRONMENT
Don’t Buy the “A.I.” Hype
“These models do not hallucinate. They make stuff up because a programmer using a random generation function decided they should…By saying they hallucinate, we are removing the responsibility of the programmer and the model trainer to be ethical.”
“[In] the most recent version of ChatGPT, in the release notes, one of the bullet points was: Less likely to make up facts…You cannot make up a fact. A fact is the truth. A fact is. You do not make up a fact. That sentence is an impossibility and shows the irresponsibility going on right now that they actually let that get out in the release notes.”
There’s More to Open Source than Code
- Non-code contributions: Any contributions that don’t involve code!
- Documentation
- Blogging
- Trying stuff out
- Issue triage
- Code reviews
- Answering/Asking questions
- Volunteering at events
- Localizing
- Organizing events
- Hosting/facilitating hackathons
- Creating assets
- Giving talks
- Video content
- Social media
- Content curation
- Moderation
- Mentorship
- Why?
- Getting out of the comfort zone
- Helping a project thrive
- Improving onboarding
- Meeting people
- As a code contributor, what can I do to facilitate these?
- Establish a safe, inclusive, effective community
- Tag issues
- Advocate for the community
- Include non-code contributions in
CONTRIBUTORS.md
Managing Content with Django
- Wagtail
- Feature rich: media management, workflow tools, polished user experience, commenting, search
- All encompassing experience. Most objects can be edited in Wagtail.
- Page-focused
- Growing community, lots of momentum
- Some batteries are hard to remove
- django CMS
- Minimalist approach, plugins recommended. “Django Admin + an editor”
- Component-first, easy to make new ones (called “plugins”)
- Heavy on frontend editing
- Smaller community, slower momentum
- Great multilingual support
- Conclusions
- Wagtail is all encompassing, getting better about gaps with Django
- Wagtail has the better UX most of the time
- Wagtail is simpler, which is a pro/con
- django CMS is great at being a small piece of a larger puzzle
- django CMS puts more emphasis on content type and structure
- django CMS frontend editing is slick for quick changes
An approach to lightweight tenancy management using Django Rest Framework
- Single Tenant: When each tenant has it’s own application/db.
- Multitenant: When each tenant shares a single application/db.
drf-nested-routers
to nest resources under each tenant.drf-access-policy
to provide role based permissions.- You can create a
TenantAwareManger
to protect against leaking data on custom queries.
What Django Deployment is Really About
- Four main concerns
- Static Files
manage.py runserver
handled this during development, but…- Once you deploy, the production web server needs to handle this (django is too slow).
- Once you’ve set
STATIC_ROOT
, you have to runmanage.py collectstatic
. - What about Whitenoise?
- Middleware that intercepts and skips slow django request for static files.
- Great when you can’t directly configure your web server.
- Excels when serving from a CDN (handles headers).
- Database
- Your local and production database setups will most likely be different.
- Generally, if you’re using a cloud provider, you’ll be connecting to a remote, managed, database.
- WSGI Server
- The thing that actually runs your Django code.
manage.py runserver
handled this during development, but…- In production, you need to pick a production WSGI Server (gunicorn/uwsgi).
- Web Server
- Apache/nginx
- VPS: Manual
- PaaS: Managed
- Static Files
- Bonus:
- Run
manage.py check --deploy
- Check out
django-simple-deploy
- Check out
django-production
- Run
Introduction to GitHub Actions: Understanding Key Terms and Building Your First GitHub Action
You configure a GitHub Actions workflow to be triggered when an event occurs that then runs a series of jobs on a runner that include one or more steps that call a script(s) or action
Day 3
Keynote: Testing Modern Web Apps Like a Champion
- Follow up to PyTexas Keynote - “Full-Stack Python”
- Testing challenges: Tests are slow, brittle, flaky, don’t make sense, don’t make money, require changing context.
- Modern testing goals
- Focus on building fast feedback loops rather than certain types of tests.
- Make test development as fast and painless as possible.
- Choose test tooling that naturally complements dev workflows.
- Writing good tests (Arrange-Act-Assert Pattern)
- Arrange things in the system
- Act on the target behavior
- Assert expected outcomes
pytest
- Web test automation tool comparison
Selenium Cypress Playwright Popular among testers Popular among developers So hot right now C#, Java, JavaScript, Ruby, Python JavaScript only C#, Java, JavaScript, Python All major full browsers Chrome, Edge, Firefox, Electron Chromium+, Firefox, WebKit WebDriver protocol In-browser lavaScript Debug protocols Just a tool - requires a framework Full, modern framework Full. modern framework Can be slow/flaky if waiting is poor Visual execution but can be slow Typically the fastest execution Open source, standards, & gov Open source from Cypress Open source from Microsoft - TLDR
- Write unit tests to catch problem directly in the code.
- Write component tests if you have a component library.
- Write API tests to grind through variations.
- Write UI tests to cover user-centric behaviors.
- Do performance and load testing to make sure the app works well.
- Run tests continuously as part of standard development workflows.
Django migrations, friend or foe? Optimize your Django migrations for faster testing
- Once you’ve built up many migration, creating your test database will start slowing down.
- You can use
--keepdb
when running tests to preserve the test database between runs, but this can be hard to take advantage of in CI/CD. - You could use
squashmigrations
, but turns out the database building performance is more or less the same. - The best solution (for local and ci/cd) is to completely rebuild your migrations.
Inside Out: My Journey of Understanding Inclusion
- Unconscious bias
- Ex: The arborist was shocked that a woman mows my lawn
- Ex: The clerk assumed I’m not operating the jigsaw
- Sometimes concepts are locked down in our minds (Some since upbringing!)
- An individual is honestly unaware of a prejudice
- It’s a twofold effort: to notice it, and then to shape it
- Source of bias can be stereotypes, preconceived notions, past experiences, or even gut instinct
- Appearance (age, [dis]abilities, gender, race, body size, etc)
- Data (title, education, language, name, religion, etc)
- Opinion (attribution, confirmation, conformity, contrast)
- We can reflect on our own unconscious biases to improve the Django community as a whole
- Double bind
- Conflicting expectations or stereotypes
- Challenging to meet both career advancement and likability criteria
- Women sometimes have to choose between being liked but not respected, or being respected but not liked
- Double bind paradoxes
- Maintaining distance yet being approachable
- Advocating for oneself yet serving others
- Authoritative yet participative
- Demanding yet caring
- Culture aspects
- What is normal in your culture, may not be in others
- We can further explore how cultural factors influences:
- being a good person
- being a good communicator
- Book: The Culture Map
- Communication context
- Some cultures use high-context (indirect, contextual)
- Some cultures use low-context (direct, explicit)
- What’s desired in one style can be seen as negative in the other
- High context could be seen as not trustworthy or lier
- Low context could be seen as condescending or patronizing
- Implicit does not (usually) work across cultures
- Perhaps is better to err on the side of verbose
- Less room for misinterpretation
- Less effort to parse from the recipient
- Also important to make explicit why we are being explicit!
- Being Convincing
- A.K.A Persuasive, in a good way!
- Some cultures use principle-first (Why, deductive reasoning)
- Some cultures use application-first (How, inductive reasoning)
- We can find a balance, and provide both
- Use TLDRs
- Communication context
- Language challenges
- Language [in]flexibility
- It’s easy (at least possible!) to speak neutrally in English
- It’s not in other languages, like in Spanish, French, Hindi, or Arabic
- Ex: In Spanish, everyone is supposed to use the masculine form to describe a group of people, but in reality the masculine form is used to describe doctors, engineers, managers, etc., while the feminine form is used to describe nurses, secretaries, teachers, etc.
- Aspirational conditioning starting from childhood
- Inclusive naming in tech
- Gender (“man-in-the-middle”)
- Disabilities (“crippled”)
- Neurodiversities (“dumb”, “crazy”, “sanity-check”)
- Groups that historically have suffered oppression (“master”, “slave”)
- Denoting negative connotations (“whitelist”, “blacklist”)
- Violent (“abort”, “kill”)
- Check your code/project/docs if you can!
- Inclusive context
- “Next spring”/“summer” applies to only half of the world!
- Abbreviations/acronyms are used for the sake of brevity or practicality but could impact new contributors
- CRUD, MTI, DDL, DTL, DDT, etc.
- Useful, but spell it out the first time it’s introduced.
- Language [in]flexibility
- Allies
- Supportive Advocate: Actively supports marginalized groups and role models for inclusive policies/practices
- Empathetic: Understands diverse experiences and challenges, and listens without judgment
- Confronts Bias: Challenges discrimination using their privilege and influence, and amplify underrepresented voices
- Self-Educates: Learns about diversity and inclusion and continuously improves as an ally
- Not self-appointed: Recognized as such by the minorities
- Conclusions
- Varied Factors: it’s a balancing act, but it’s worth the effort
- No Magic Formula: no guaranteed recipe or procedure exists
- Acknowledge Challenges: the first step toward resolution
- Reflect and Act: introspect as individuals and as a group
- Pursue Equity: evaluate opportunities for change and improve
- Provide support: allies are key in the process of change
Lightning Talks - Day 3
- Starting a business in 5 minutes
- You will fail, but some things will succeed…this is called “finding your niche”
- It will be expensive, so keep it simple
- Wait to hire your first employee…until you have 3x their salary
- Everything is people, and people needs systems
- If you build it, they will not come…get your first customer before launching
- You can do it!
- Strategies for a long running side projects
- Doc…you…men…tay..shun……Documentation!
- Tests…what the code should and should not do.
- Issues…write up your features as issues.
- Code comments
- How to get into dev mentorship
- The only true measure of success is the number of people you have helped times the effect of that help.
- People in a position of privilege can open doors and make opportunities
- Humility: An unexpected treatment for imposter syndrome
- Humility provides honest self-awareness and evaluation to contradict doubts.
- Humility encourages others around you.
- Humility is a discipline for everyone.
Best Practices for Making a Wagtail Site as Accessible as Possible
- Accessibility is practice of ensuring our web content is able to be consumed by the widest possible spectrum of people.
- This is a legal responsibility for many…and a moral responsibility for all.
- Wagtail has two main accessibility initiatives:
- Accessibility of Wagtail itself
- Accessibility of sites it produces (this talk)
- Wagtail comes with an accessibility checks that allows editors to identity and correct accessibility issues themselves.
- What is still up to developers
- Semantic markup in templates
- Configuring Wagtail’s accessibility checker
- Default is to only show errors that an editor can address
- Configure to show all errors to superusers
- Enforcing proper heading hierarchy
- You could override blocks to validate
- Promoting good alt text
- Wagtail doesn’t have alt-text on the image model because alt text should be contextual
- You can create a custom block that ties an image to alt text
- Using ARIA labels, where appropriate
- Helping editors help themselves
- Use
help_text
- Create a
HelpPanel
- Use
One database table, one model, many behaviours: Proxy model
Proxy models allows you to create a new model based on an existing model without creating a new database table, can be used to modify the behavior of the original model.
Back to the Future of Hypermedia in Django
- REST: Representational State Transfer
- HATEOAS: Hypermedia as the Engine of Application State
- The server controls everything, the client just displays the result.
- Better for accessibility and sustainability.
- Not HATEOAS: JSON as the Engine of Application State
- Great for complex interdependencies and/or offline apps, but overkill for general CRUD apps.
- Common patterns and resources
Django’s accessibility track record
- 96.3% of the top 1,000,000 homepages have basic accessibility issues
- ~95% of a sample of 25,000 Django homepages have basic accessibility issues
- Accessibility Insights Extension
Panel Discussion: Who put me in charge? Moving beyond day-to-day coding in Django
“What’s important for you as a managers who is managing devs is to understand what they’re working on” – Julia
“I don’t know if it’s as much a skill [switching from contributor to manager] as it is like being genuinely interested in the people that work for you” – Elizabeth
“Sometimes…I can sort of see a pothole coming, you know, either a technical pothole or maybe a political one or maybe a legal one or a financial one, and say ok, there’s a pothole coming up” – Charles
“Your input is not just being able to directly say ‘ok this project might be a problem.’ It’s when you get other people to the point where they can see the project may be a problem.” – Charles
“You do lose a little bit of political sway [returning to an individual contributor].” – Andrew
“A good technical leader breaks a plan down to 3 month chunks.” – Andrew
“How do you identify some of those initial signals that you’re ready to take that step to [becoming a manager]?” – Jay
“It’s similar to the question: when are you ready to have children?” – Charles
Reply via email“Being really good at documentation has always benefitted me in my career and particularly in management.” – Elizabeth