An end-to-end PostgreSQL to Microsoft SQL Server migration I built on a CMS Medicare provider and services dataset. I assessed the source database, converted the full schema — including a custom audit-logging trigger — to native T-SQL, migrated 20.5 million rows with a custom Python ETL pipeline after discovering Microsoft's migration tool no longer supports PostgreSQL as a source, validated the result across three layers, and benchmarked and tuned performance against the original system.
I wanted to build a migration project that covers the full scope of what moving a production database actually requires — not just copying tables. Assessment, schema conversion, data migration, validation, and performance tuning aren't isolated steps. They form a complete discipline, and this project treats them that way.
I used a CMS Medicare provider and services dataset already running in PostgreSQL 18 — 20.5 million rows across 5 tables in two schemas (core provider data and audit logging) — to simulate a real production migration to Microsoft SQL Server 2025, built on a Hyper-V VM alongside the source system.
Every table matched the source exactly after migration — including a clean, zero-row audit log
I inventoried every table, column, constraint, index, trigger, and sequence in the source PostgreSQL database across two schemas, flagged PostgreSQL-specific types with no direct SQL Server equivalent, and discovered mid-project that Microsoft's SSMA no longer supports PostgreSQL as a source — so I pivoted to a custom Python ETL instead.
Phase CompleteI converted every table, key, and index across two schemas to T-SQL, mapped PostgreSQL-specific types to their SQL Server equivalents, and rewrote a generic PL/pgSQL audit-logging trigger — shared across multiple tables via dynamic metadata — as two dedicated T-SQL triggers using the native JSON type and FOR JSON PATH.
Phase CompleteI built a Python ETL pipeline that migrated 20.5 million rows in the correct referential order, disabling the audit triggers during the bulk load to avoid flooding the audit table with migration noise, then re-enabled them and reseeded the identity counter — validating zero rows leaked into the audit log afterward.
Phase CompleteI validated the migration across three layers: aggregate checksums matching exactly between source and target, four constraint-enforcement tests confirming SQL Server rejects bad data the same way PostgreSQL did, and three representative queries returning identical results on both systems.
Phase CompleteI baselined six representative queries against both systems, then investigated a real regression: SQL Server ran one join nearly 2× slower than PostgreSQL. Four different tuning attempts all confirmed the same conclusion: a genuine structural bottleneck, not a missing index.
Phase CompleteI documented every phase as I completed it — architecture, decisions, real errors and how I fixed them, and results — and packaged the project into this standalone, presentable portfolio piece with full source code and commit history on GitHub.
Phase CompleteI built the entire environment on a single Windows 11 Pro laptop using Hyper-V, rather than relying on a pre-existing SQL Server install. A Windows Server 2025 VM hosts SQL Server 2025 Enterprise Developer Edition, networked to my existing native PostgreSQL 18 installation on the host. I chose this VM deliberately as the future Always On Availability Group primary replica for a follow-on Enterprise DBA project, so the environment serves double duty rather than being rebuilt later.
Getting there wasn't friction-free, and I documented the real troubleshooting rather than smoothing it over: TCP/IP was disabled by default in SQL Server Configuration Manager, no Windows Firewall rule existed for port 1433, and a dedicated etl_service login needed specific ALTER permissions before it could even disable my own audit triggers for the bulk load.
SSMS 21 connected to SQL Server 2025 Enterprise Developer Edition on the Hyper-V VM
The custom Python ETL pipeline migrating all four tables — ~20.5 million rows in under 15 minutes
I baselined six representative queries — covering point lookups, joins, aggregations, filtering, and a heavy join-plus-sort — against both systems. The results below are real measured execution times from psql and SSMS, not estimates.
| Query | Description | PostgreSQL | SQL Server | Result |
|---|---|---|---|---|
| Q1 | Point lookup by primary key | 13.2 ms | 50 ms | PostgreSQL faster |
| Q2 | Join + filter by state | 1,969.7 ms | 3,582 ms | Investigated in depth |
| Q3 | Group-by aggregation by provider type | 162.2 ms | 109 ms | 33% faster on SQL Server |
| Q4 | Join + group-by aggregation by state | 1,515.0 ms | 967 ms | 36% faster on SQL Server |
| Q5 | Filtered range scan | 467.3 ms | 316 ms | 32% faster on SQL Server |
| Q6 | Heavy join + aggregation + sort | 6,096.4 ms | 2,051 ms | 66% faster on SQL Server |
Q2's confirmed bottleneck — an Index Scan across 9,660,628 rows, 77% of query cost, after four different tuning attempts
psycopg2 and pyodbc — a more hands-on but equally valid path.providers and provider_services. I disabled both triggers before the bulk load and re-enabled them immediately after, confirmed by a clean, zero-row audit table post-migration.state_abbr to filter on), not a missing index. I documented that conclusion rather than forcing an artificial fix.Full source code — all SQL scripts and Python files organized by phase, complete documentation for every phase, and commit history showing every step of the migration from day one.
View SourceSee the full portfolio — including the PostgreSQL DBA project, the MySQL DBA project, and other data and database projects built on real public datasets.
View All Projects