Best ETL tools for small teams: what you can skip
Choosing data tooling without a data engineer: what a small team genuinely needs, what the enterprise stack assumes you have, and the cheapest arrangement that still works.
Osman Ramadan11 min read
On this page
Most writing about data tooling assumes a data team. It describes a stack with an ingestion layer, an orchestrator, a transformation framework, a catalog, a quality tool and an observability platform, which is sound advice for a company with three data engineers and ruinous advice for a company with none.
This page is for the second case: what a small team actually needs, what the standard stack assumes you have that you do not, and the arrangement that costs least while still producing numbers you can trust. For the full landscape by layer, see the data pipeline tools guide.
What we'll cover
The question before the tools
What decision is this data supporting, and who makes it?
Small teams routinely build a warehouse because it seems like the responsible thing to do, and then nobody queries it. The first thing to establish is which recurring question you are trying to answer reliably, because that determines how much machinery is justified.
Three honest answers, each pointing somewhere different.
"We just want our numbers in one place once a week." You may not need a warehouse. Scheduled exports into a spreadsheet, or your existing tools' own reporting, may cover it entirely. This is a genuine answer, not a failure.
"We have three systems that disagree and we need one version." Now you need somewhere to bring them together, which is a warehouse, and you need transformation to reconcile them. This is where a small stack starts paying.
"We want to analyse behaviour across the product and the CRM." A real analytics need, and worth building properly.
Establishing this first prevents the common outcome where a small team builds a pipeline that works and answers nothing anyone asked.
What a small team can skip
The standard stack has pieces that exist to solve problems you do not yet have.
Skip the orchestrator. Airflow, Dagster and Prefect solve dependency management across many jobs. With four jobs in a fixed order, your warehouse's own scheduler or a cron job is sufficient and vastly easier to understand.
Skip the data catalog. Catalogs help people find data across a large organization. With six people, asking a colleague is faster and more accurate.
Skip the observability platform. Dedicated data observability is worth it at scale. A handful of assertions in your transformations covers the small-team version of the same need.
Skip streaming entirely. Unless a decision depends on data seconds old, batch is correct and dramatically simpler.
Skip the reverse ETL tool initially. Pushing warehouse data back into operational systems is useful and is usually better served early on by a simple automation than by another platform.
Do not skip version control on your transformations. This is the one piece of apparent ceremony that pays immediately, because it is what lets you answer "why did this number change" three months from now.
The minimum viable stack
Four pieces, and the whole thing can cost under a hundred dollars a month at small volumes.
A warehouse. BigQuery, Snowflake, or Postgres. For a genuinely small team, Postgres is frequently enough and you may already run one. BigQuery's pricing suits bursty analytical querying and its free tier covers real exploration.
Ingestion. Covered in the next section, and the piece with the widest cost range.
Transformation in SQL, version-controlled. dbt Core is free, runs from your repository, and gives you dependencies, testing and documentation. This is the highest-value piece for a small team because it converts a pile of ad-hoc queries into something reviewable.
Somewhere to look at it. Metabase, Looker Studio, or your warehouse's own console. Looker Studio is free and connects to most warehouses; Metabase is pleasant and can be self-hosted.
Scheduling via your warehouse's scheduler, dbt Cloud's scheduler, or a cron job. Not an orchestration platform.
That is a complete, honest analytics stack. It is not impressive and it answers questions correctly, which is the entire point.
Ingestion options and what they cost
The piece where small teams most often overspend, and the one with genuinely good cheap options.
Write your own scripts. For one to three stable sources with decent APIs, this is a few hours each and costs nothing ongoing. The risk is maintenance: when the API changes, it is yours. For stable sources, that is rarely.
Airbyte self-hosted. No licence cost, a large connector catalog, and you run the deployment. Reasonable if someone is comfortable with containers; a recurring obligation if nobody is.
Managed connectors — Fivetran, Airbyte Cloud, Stitch, Hevo. You pay, and connector maintenance stops being yours. The pricing is usually per row or record, which is worth modelling carefully because a chatty source or an accidental full resync can produce a startling bill.
Your tools' own scheduled exports. Underrated. Many SaaS products will email a CSV or drop one on a schedule, and picking that up is simple and free. For a small team this covers more sources than people expect.
The decision rule: count your sources and how often their APIs change. One or two stable sources, write scripts. Four or more, or anything with a volatile API, pay for managed ingestion, because you are buying maintenance rather than code.
Before committing to a paid plan, model the cost at your actual row volume and check what a full resync costs. That is the number that surprises people, and it usually happens during the first month when you are still getting the configuration right.
The spreadsheet problem
Every small team has a spreadsheet that is secretly production infrastructure. Somebody maintains it by hand, several decisions depend on it, and it exists outside every system you have just built.
Two mistakes are available here.
Pretending it does not exist, building a warehouse alongside it, and ending up with two versions of the numbers that disagree. This is the common outcome and it is worse than either alternative on its own.
Migrating it immediately, which fails because the spreadsheet encodes judgement nobody has written down — exceptions, manual corrections, a column someone overrides when they know better.
The workable approach is to bring it in rather than replace it. Sync the spreadsheet into the warehouse as a source, so the manual judgement it contains is available alongside the automated data. Then, over time, move the parts that turn out to be rules rather than judgement.
Keeping that sync current is a small recurring automation rather than a data engineering project. On CodeWords you describe it in plain language — which sheet, how often, where it lands, what to do when a column changes — and Cody, the automation builder, builds it, connects it to your systems, and deploys it. Automations connect to more than 3,000 integrations. The free plan covers light use, with Pro at $39 per month and Business at $100 per month as usage grows; details are on the pricing page.
Trusting the numbers without a data team
A small team has no analytics engineer to catch a silently wrong number, so the checks have to be built in.
Assert on row counts. A table that normally gains a few hundred rows a day and gained four has a problem, and no failed job will tell you.
Assert on nulls in key columns. Sudden nulls in a joining key is the signature of an upstream schema change.
Reconcile totals against the source. Revenue in the warehouse against revenue in the billing system, weekly. This catches more real problems than every other check combined, and it is the one small teams skip.
Check freshness. When did this table last update? A stale table looks identical to a current one in a chart.
Report success, not only failure. A short weekly message saying the pipeline ran, the counts, and the reconciliation result makes silence meaningful. Without it, nobody can tell a quiet week from a broken pipeline.
dbt's built-in tests cover most of this in a few lines each, which is why it earns its place ahead of tools that look more impressive.
When to add the next piece
Add an orchestrator when the order of your jobs starts mattering and you have been caught by a transformation running before its data arrived.
Add managed ingestion when you have fixed the same broken connector twice.
Add a BI tool properly when people outside the team need to answer their own questions rather than asking you.
Add a data person when the answer to "who owns this" has been a shrug for three months. That is the real signal, and it arrives before the tooling signals do.
Resist adding anything because an article said a modern stack has it. The stack described here answers questions correctly, and correctness is the product.
Frequently asked questions
Do we need a data warehouse at all?
Not necessarily. If your question is answerable inside one system's reporting, use that. You need a warehouse when several systems disagree, or when a question spans systems that cannot see each other. Building one before that point is common and usually idle.
Is Postgres good enough as a warehouse?
For a small team, frequently yes, particularly if you already run one. Purpose-built warehouses win on large analytical scans and separated compute, and neither matters much at small data volumes. Starting on Postgres and moving later is a reasonable path.
dbt Core or dbt Cloud?
Core is free and runs anywhere, including from a scheduled job. Cloud adds a scheduler, an interface and collaboration features. For a small team, Core with your existing scheduling is usually sufficient, and Cloud is worth it when several people are working on models.
How much should a small team's data stack cost?
Under a hundred dollars a month at small volumes is achievable: a modest warehouse, dbt Core at no cost, a free BI tool, and either self-written ingestion or a small managed plan. Costs rise with row volume on ingestion and with query volume on the warehouse, in that order.
What is the most common mistake?
Building the pipeline before agreeing the question. The result works, is technically sound, and answers nothing anybody asked. Start from one recurring decision and build the smallest thing that supports it.
Should we hire or outsource this?
Neither, at first. The stack above is within reach of a technically comfortable generalist. Bring in help when you have a specific problem that is costing you, rather than to build the stack itself, because a stack built by someone who then leaves is a liability.
How do we handle personal data?
Decide before it lands rather than after. Filter or mask at ingestion where regulation requires it, since loading raw personal data into the warehouse and cleaning it there means it was in the warehouse. This is the one case where transforming before loading is still the right order.
What if our data lives in tools with no API?
More common than the tooling landscape admits. Most such products will email a scheduled CSV or drop one on a shared folder, and picking that up is simple, free and surprisingly durable. Screen scraping is the last resort and breaks on someone else's schedule; a manual monthly upload into a known location, with everything automated around it, beats a fragile scraper.
How often should the pipeline run?
Daily is right for most small-team analytics, and overnight is usually fine. Move to hourly only when somebody is genuinely waiting on fresher numbers, and remember that more frequent runs cost more on row-based ingestion pricing. Start daily and let a real complaint move you.