The construction industry is traditionally considered slow to adopt digital technologies. However, it's an area ripe with data, especially unstructured video, documents, and logs. At Real Construction, I worked on backend systems that transformed chaotic data streams into actionable insights and regulatory compliance.
Problems Worth Solving
- Unstructured, video-based site footage not indexed or searchable
- Contractor documents subject to regional standards
- Internal tooling lacked role-based access control (RBAC)
We set out to change this using Python, FastAPI, Docker, and PostgreSQL.
Secure Access via RBAC
First, we built a fine-grained permission system using FastAPI and PostgreSQL. Engineers, auditors, and contractors had different access scopes.
Security Features:
- JWT-based auth
- Granular roles (e.g., "Site Supervisor" vs "Regulatory Auditor")
- SQLAlchemy policies for row-level data filtering
This system allowed us to safely expose internal tools to different stakeholders without compromising sensitive information.
Automating Compliance: Rule Engines in Python
To validate contractor documents against regulatory standards, we developed a Python-based rule engine.
Capabilities:
- Read document metadata and content
- Apply evolving JSON rulesets
- Validate expiration dates, certifications, and formatting
This engine ran nightly over new uploads and flagged inconsistencies before they became liabilities.
Working with Video: Dockerized Pipelines
Construction sites often use time-lapse cameras or drones. We built a video ingestion and indexing pipeline:
- Ingest footage into S3-compatible storage
- Extract frames and embed metadata (timestamp, location)
- Serve processed assets via lightweight REST API
We containerized this pipeline for consistent deployment across edge and cloud environments.
Deploying AFFiNE + CI/CD
For cross-team documentation and collaboration, we deployed AFFiNE in a secure, self-hosted environment. Our CI/CD pipeline ensured:
- Versioned documentation updates
- Markdown-first, user-editable guides
- Access control using OAuth
This became our internal wiki, integrated into onboarding and compliance processes.
ML-Adjacent, Not ML-Dependent
Though we didn't deploy deep models here, our pipeline was ML-friendly:
- Preprocessed data for future computer vision analysis
- Structured logs for anomaly detection models
- Rule engines that could be augmented with classifiers
We laid the foundation for future ML integration.
Conclusion
Construction tech needs more than flashy dashboards—it needs robust backends that process real-world, unstructured data. Our work focused on enabling that. With automation, access control, and modular pipelines, we brought intelligence to an industry that's just beginning its digital journey.