Automation and data cleanup
Most organizations have someone who redoes the same data work every week: export, reformat, re-key, reconcile. I clean and migrate what already exists, then move the recurring part into a Python job or a Postgres function, so the numbers stop drifting and nobody has to remember to run it.
How I approach the work
A spreadsheet is often the right answer, and I have talked people out of this before. If one person owns the file and the format changes as often as the work does, automation buys you rigidity instead of time. What changes the math is repetition across people: the same export re-keyed every week, two systems that disagree about the same customer, or a file that quietly became the system of record for something that matters.
An engagement starts with me watching the real process, including the steps nobody wrote down. Then I clean and migrate the existing data, because automating a mess only produces the mess faster. The recurring piece gets built last: a Python job, a Postgres function, or a small internal screen, with a written handoff so you are not dependent on me to run it. Scope is fixed per project, and the number lands after the audit rather than before it. See how pricing works.
I do this at scale. GrantCue, the grant-funding SaaS I built, rebuilds its catalog on a schedule from the full federal Grants.gov corpus plus more than 100 state and state-agency sources, normalized through per-source adapters, deduplicated by content hash and title similarity, then upserted only where something changed. WellStockd applies the same discipline in a kitchen: cooking a recipe atomically subtracts its ingredients from inventory, and anything that runs out lands on the shopping list. Those kitchen flows run as Postgres functions, so the numbers never drift.
When the data only lives on a website, extraction is still solvable. I have built a multi-tenant web crawler and SEO auditor in Python and Flask with Playwright rendering for exactly that case. I work from Lock Haven, though this kind of job runs remotely more often than not; the data does not care where either of us sits.
What you get
- A written map of the current process, including the manual steps and the places where two sources already disagree
- A cleaned, deduplicated dataset with the matching rules written down where you can read them
- Data migration into Postgres, Supabase, or whatever system is meant to hold it, with the old source left readable until you sign off
- Scheduled import and export jobs in Python, with retries, run logs, and an alert when a run fails
- A spreadsheet-to-database move that can still hand back a spreadsheet view for the people who want one
- Small internal screens for the steps a person genuinely has to review or approve
- Extraction from a website or another vendor's tool when there is no export and no API worth using
- A runbook you can hand to someone else, including how to turn the whole thing off
Project fit
Fit. For the office where one person loses a morning a week moving the same data between two systems by hand, and the errors that slip through cost something.
Engagement. Priced after the audit, not before it. The number of source systems and the condition of the input data do most of the moving.
How a project runs
- Map the work. We sort the real workflow from the wish list, identify users, and write down what success needs to look like.
- Scope the first release. You get a clear proposal, timeline, and build plan with the risky parts pulled forward instead of hidden at the end.
- Build in the open. I share progress, ask focused questions, and keep the work testable so feedback arrives while it can still shape the product.
- Launch and hand off. Deployment, documentation, and training are part of the work, so the finished system is something your team can actually run.
Shipped work behind this
- GrantCue — A multi-tenant platform that syncs Grants.gov and 100+ state sources on a schedule, matches each organization to grants and to foundations that fund work like theirs, and drafts proposals in a team pipeline.
- WellStockd — A touch-first recipe and fridge-inventory app that lives on a fridge-mounted tablet. Cook a recipe and the ingredients subtract themselves; run out of something and it lands on the shopping list.
Related articles
Related services
Common questions
What gets cleaned before anything is automated?
The existing data comes first, because automating a mess only produces the mess faster. I start by watching the real process, including the steps nobody wrote down, and mapping where two sources already disagree. Then I deduplicate and migrate the data, with the matching rules written down where you can read them, and the old source stays readable until you sign off. The recurring job gets built last, once the data it runs on is clean.
Can you move a spreadsheet into a database?
Yes. A spreadsheet-to-database move takes the cleaned data into Postgres, Supabase, or whatever system is meant to hold it, and the people who prefer a spreadsheet can still get a spreadsheet view back. The old file stays readable until you sign off on the new one. Where a person genuinely has to review or approve a step, I build an internal screen for that step instead of leaving it in a shared file.
How is an automation project priced?
An automation project is priced after an audit of the current process, not before it. The number of source systems and the condition of the input data do most of the moving, and neither shows until someone has looked at the real files. The audit produces a written map of the process, and the fixed number for the project follows from it. Nothing gets built until that number is in writing.
What happens when a scheduled job fails?
A failed run sends an alert instead of failing silently. I write scheduled import and export jobs in Python with retries and run logs, and the handoff includes a runbook you can give to someone else, including how to turn the whole thing off. GrantCue applies the same discipline to its twenty-plus scheduled jobs, which a distributed lock keeps from colliding with each other.
Can you pull data from a website that has no export?
Usually, yes. When the data only lives on a website or inside another vendor's tool, and there is no export or API worth using, extraction is still solvable. I built a multi-tenant web crawler and SEO auditor in Python and Flask, with Playwright rendering, for exactly that case. GrantCue's grant catalog is rebuilt on a schedule from public grant portals through per-source adapters.
Start a project
Bring me the practical version.
Tell me what you're trying to build or improve and what's already in place — I'll reply with a clear next step.