Building a Deployment Pipeline for My Tools Site: From Local Windows to DreamHost
Setting up a web application often involves a messy cycle of FTP uploads and manual terminal commands. This evening, I moved beyond that by building a professional Git-based deployment pipeline. This setup allows me to code locally on Windows and deploy to my DreamHost server with a single git push.
The Architecture
The system uses a "Three-Point" sync strategy: 1. Local Machine (Windows): Where the coding happens in VS Code. 2. Bare Repository (The Storage Tank): A central Git repo on the server that acts as the deployment trigger. 3. Live Directory: The public-facing folder where the website actually runs.
1. Establishing the SSH Connection
To avoid the "infinite password loop" in VS Code, I moved from password authentication to SSH Key Pairs.
By generating an RSA key on Windows and adding it to the server's authorized_keys, I established a secure, password-less handshake. This is the foundation for automated Git commands.
2. Setting Up the Git Bridge
I initialized a Bare Repository on the DreamHost server. Unlike a standard repo, a bare repo has no working files; it only stores the version history. This makes it the perfect "middle-man" for deployment.
I then linked my local project to this server repo.
git remote add dreamhost ssh://dh_jfqmkg@iad1-shared-e1-27.dreamhost.com/~/tools_repo.git
3. Handling the "Heavy" Virtual Environment
One major hurdle was the Resource temporarily unavailable error. This happened because Git was trying to track the venv folder, which contains thousands of files.
The Fix: 1. I created a .gitignore to exclude the virtual environment, environment secrets, and cache folders. 2. I moved the responsibility of package management to a requirements.txt file.
This kept the Git transfers lightweight and prevented the server from crashing due to thread limits.
4. The Magic: post-receive Automation
The centerpiece of this setup is the Git Server-Side Hook. I wrote a script that triggers automatically the moment code is pushed to the server.
The script does three things: 1. Checkouts the code into the live web folder. 2. Auto-Installs dependencies using the requirements file. 3. Restarts the application by touching the FastCGI file.
Hook Script
#!/bin/bash
# Define the target directory for the live site
TARGET="/home/dh_jfqmkg/tools.dandevins.com"
# Define the git directory
GIT_DIR="/home/dh_jfqmkg/tools_repo.git"
# 1. Deploy the files to the web folder
git --work-tree=$TARGET --git-dir=$GIT_DIR checkout -f main
# 2. Automatically install any new Python requirements
$TARGET/venv/bin/pip install -r $TARGET/requirements.txt
# 3. Restart the app (Touch the FCGI file)
touch $TARGET/app.fcgi
echo "Deployment successful: Files updated and requirements installed."
The New Developer Workflow
Now, my development cycle is clean and efficient: 1. Code: Edit files in VS Code on Windows. 2. Commit: Save changes locally via Git. 3. Deploy: Push to the DreamHost remote.
The terminal handles the rest, and the site updates in seconds. No FTP, no manual restarts, and no resource errors.