Bell Canada — Data Engineer
June 2025 - PresentI own production data systems that millions depend on every day.
130,000+records recovered
83%faster queries
~3.5Mduplicate rows eliminated
- 01DetectAnomaly detected in record counts and query latency.
- 02TraceTraced lineage across pipelines, logs, and upstream sources.
- 03RepairRepaired transformations and backfilled missing partitions.
- 04ReconcileValidated counts and reconciled datasets end-to-end.
- 05MonitorAdded data quality checks and latency monitoring.
- Enforce exact row-count parity before every downstream step.
- Reverse-engineer undocumented legacy logic instead of stalling on poor docs.
- Rebuild fragile reporting views into maintainable, validated ETL.
- Restore data trust by recovering lost history and validating the backfill.
- Treat throughput, correctness, and observability as one reliability problem.
- Separate technical feasibility from access and process blockers.
- 01
# Incident: rows multiplying on load - 02
# Runbook: dedup_recovery (generalized) - 03
- 04
1. Confirm the duplication - 05
- Count distinct vs total per key - 06
- Sample the shape of the dupes - 07
2. Find where it enters - 08
- Trace lineage to the load step - 09
- Diagnose the load-key mismatch - 10
3. Re-key the load - 11
- Upsert key vs primary index - 12
- Make the write idempotent - 13
4. Prove it before promoting - 14
- Idempotency: run it twice - 15
- Reconcile counts end to end - 16
5. Stage the cutover - 17
- Rehearse a reversible plan - 18
- Monitor after go-live
Before
- Dynamic runtime computation
- Sequential event sequencing
- Hardcoded 30-day lookback
- Non-idempotent retries
- Monolithic SQL
After
- Materialized pre-aggregation
- Two-pass group-by propagation
- SCD Type 2 temporal joins
- Composite upsert idempotency
- Zero-downtime abstraction layer
Same 23M-row result set. 12 min → 2 min. 83% faster.
| Area | I own | I partner | I support |
|---|---|---|---|
| Data Pipelines | ✓ | ||
| Data Quality | ✓ | ||
| Incident Response | ✓ | ||
| Backfills & Recovery | ✓ | ||
| Performance Tuning | ✓ | ✓ | |
| Stakeholder Comms | ✓ | ✓ | |
| Monitoring & Alerting | ✓ | ✓ | |
| Documentation | ✓ | ✓ | ✓ |