updated readme
This commit is contained in:
+346
-230
@@ -1,277 +1,393 @@
|
|||||||
Git-Sync Mirror Agent
|
**Git-Sync Mirror Agent**
|
||||||
Git-Sync is a containerized mirroring agent designed to watch a local Git repository and propagate changes to multiple remote repositories with high reliability, auditability, and fault recovery. It supports mirroring to platforms like GitHub, Forgejo, Radicle, Internet Archive, and Web3.storage, ensuring your repository is resiliently backed up across diverse infrastructure. Git-Sync is a key component of the fold-stack project, emphasizing sovereignty and fieldcraft-resilient design.
|
|
||||||
📜 Overview
|
**Git-Sync** is a containerized mirroring agent designed to watch a local Git repository and propagate changes to multiple remote repositories with high reliability, auditability, and fault recovery. It supports mirroring to platforms like GitHub, Forgejo, Radicle, Internet Archive, and Web3.storage, ensuring your repository is resiliently backed up across diverse infrastructure. Git-Sync is a key component of the fold-stack project, emphasizing sovereignty and fieldcraft-resilient design.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
📜 **Overview**
|
||||||
|
|
||||||
Git-Sync operates as a Docker container that:
|
Git-Sync operates as a Docker container that:
|
||||||
Watches a local Git repository for changes.
|
|
||||||
Syncs changes to multiple configured remotes (GitHub, Forgejo, Radicle, Internet Archive, Web3.storage).
|
* Watches a local Git repository for changes.
|
||||||
Ensures atomicity using lockfiles.
|
* Syncs changes to multiple configured remotes (GitHub, Forgejo, Radicle, Internet Archive, Web3.storage).
|
||||||
Logs all operations for auditability.
|
* Ensures atomicity using lockfiles.
|
||||||
Implements failure recovery with retries and exponential backoff.
|
* Logs all operations for auditability.
|
||||||
Operates independently of commercial SaaS infrastructure.
|
* Implements failure recovery with retries and exponential backoff.
|
||||||
Supported Remotes
|
* Operates independently of commercial SaaS infrastructure.
|
||||||
GitHub: Push via SSH.
|
|
||||||
Forgejo: Push via SSH to a self-hosted instance.
|
**Supported Remotes**
|
||||||
Radicle: Peer-to-peer Git (placeholder, requires rad CLI implementation).
|
|
||||||
Internet Archive: Git bundles uploaded via Rclone.
|
* **GitHub**: Push via SSH.
|
||||||
Web3.storage: Git bundles uploaded via Rclone (optional).
|
* **Forgejo**: Push via SSH to a self-hosted instance.
|
||||||
Extendable: Can be extended to support S3, IPFS, Sia, etc.
|
* **Radicle**: Peer-to-peer Git (placeholder, requires rad CLI implementation).
|
||||||
🛠️ Prerequisites
|
* **Internet Archive**: Git bundles uploaded via Rclone.
|
||||||
|
* **Web3.storage**: Git bundles uploaded via Rclone (optional).
|
||||||
|
* **Extendable**: Can be extended to support S3, IPFS, Sia, etc.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
🛠️ **Prerequisites**
|
||||||
|
|
||||||
Before setting up Git-Sync, ensure you have the following:
|
Before setting up Git-Sync, ensure you have the following:
|
||||||
Docker and Docker Compose installed.
|
|
||||||
Install Docker: Official Docker Installation Guide
|
* **Docker** and **Docker Compose** installed.
|
||||||
Install Docker Compose: Official Docker Compose Installation Guide
|
* Install Docker: [Official Docker Installation Guide](https://docs.docker.com/get-docker/)
|
||||||
Git installed for managing the local repository.
|
* Install Docker Compose: [Official Docker Compose Installation Guide](https://docs.docker.com/compose/install/)
|
||||||
Install Git: Official Git Installation Guide
|
* **Git** installed for managing the local repository.
|
||||||
A local Git repository to mirror (e.g., volumes/repos in fold-stack).
|
* Install Git: [Official Git Installation Guide](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git)
|
||||||
SSH keys for GitHub and Forgejo.
|
* A local Git repository to mirror (e.g., volumes/repos in fold-stack).
|
||||||
Rclone configured for Internet Archive and Web3.storage (if using these remotes).
|
* SSH keys for GitHub and Forgejo.
|
||||||
A machine with at least 2GB of RAM (Git-Sync is lightweight but depends on fold-stack requirements).
|
* Rclone configured for Internet Archive and Web3.storage (if using these remotes).
|
||||||
🚀 Setup Instructions
|
* A machine with at least 2GB of RAM (Git-Sync is lightweight but depends on fold-stack requirements).
|
||||||
1. Clone the fold-stack Repository
|
|
||||||
|
---
|
||||||
|
|
||||||
|
🚀 **Setup Instructions**
|
||||||
|
|
||||||
|
**1\. Clone the** fold-stack **Repository**
|
||||||
|
|
||||||
Git-Sync is part of the fold-stack project. Clone the repository if you haven’t already:
|
Git-Sync is part of the fold-stack project. Clone the repository if you haven’t already:
|
||||||
|
|
||||||
bash
|
bash
|
||||||
|
|
||||||
git clone https://github.com/mrhavens/fold-stack.git
|
git clone https://github.com/mrhavens/fold-stack.git
|
||||||
cd fold-stack
|
cd fold-stack
|
||||||
2. Initialize a Local Git Repository
|
|
||||||
|
**2\. Initialize a Local Git Repository**
|
||||||
|
|
||||||
Git-Sync watches a local repository at volumes/repos. Initialize it if it doesn’t exist:
|
Git-Sync watches a local repository at volumes/repos. Initialize it if it doesn’t exist:
|
||||||
|
|
||||||
bash
|
bash
|
||||||
mkdir -p volumes/repos
|
|
||||||
|
mkdir \-p volumes/repos
|
||||||
cd volumes/repos
|
cd volumes/repos
|
||||||
git init
|
git init
|
||||||
echo "# Test Repo" > README.md
|
echo "\# Test Repo" \> README.md
|
||||||
git add .
|
git add .
|
||||||
git commit -m "Initial commit"
|
git commit \-m "Initial commit"
|
||||||
git branch -M main
|
git branch \-M main
|
||||||
cd ../..
|
cd ../..
|
||||||
3. Generate SSH Keys for GitHub and Forgejo
|
|
||||||
|
**3\. Generate SSH Keys for GitHub and Forgejo**
|
||||||
|
|
||||||
Git-Sync uses SSH keys to authenticate with GitHub and Forgejo.
|
Git-Sync uses SSH keys to authenticate with GitHub and Forgejo.
|
||||||
3.1 Generate SSH Key for GitHub
|
|
||||||
|
**3.1 Generate SSH Key for GitHub**
|
||||||
bash
|
bash
|
||||||
ssh-keygen -t ed25519 -C "your_email@example.com" -f ~/.ssh/github_key
|
|
||||||
3.2 Generate SSH Key for Forgejo
|
ssh-keygen \-t ed25519 \-C "your\_email@example.com" \-f \~/.ssh/github\_key
|
||||||
|
|
||||||
|
**3.2 Generate SSH Key for Forgejo**
|
||||||
bash
|
bash
|
||||||
ssh-keygen -t ed25519 -C "your_email@example.com" -f ~/.ssh/forgejo_key
|
|
||||||
Press Enter to skip setting a passphrase, or set one for added security.
|
ssh-keygen \-t ed25519 \-C "your\_email@example.com" \-f \~/.ssh/forgejo\_key
|
||||||
This creates ~/.ssh/github_key and ~/.ssh/forgejo_key (private keys) and their .pub counterparts (public keys).
|
|
||||||
4. Configure SSH Keys on GitHub and Forgejo
|
* Press Enter to skip setting a passphrase, or set one for added security.
|
||||||
4.1 Add SSH Key to GitHub
|
* This creates \~/.ssh/github\_key and \~/.ssh/forgejo\_key (private keys) and their .pub counterparts (public keys).
|
||||||
Copy the public key:
|
|
||||||
bash
|
**4\. Configure SSH Keys on GitHub and Forgejo**
|
||||||
cat ~/.ssh/github_key.pub
|
|
||||||
Go to GitHub > Settings > SSH and GPG keys > New SSH key.
|
**4.1 Add SSH Key to GitHub**
|
||||||
Title: fold-stack-git-sync.
|
|
||||||
Key type: Authentication Key.
|
* Copy the public key:
|
||||||
Key: Paste the public key.
|
* bash
|
||||||
Click Add SSH key.
|
* cat \~/.ssh/github\_key.pub
|
||||||
Test the connection:
|
* Go to [GitHub](https://github.com/) \> **Settings** \> **SSH and GPG keys** \> **New SSH key**.
|
||||||
bash
|
* Title: fold-stack-git-sync.
|
||||||
ssh -i ~/.ssh/github_key -T git@github.com
|
* Key type: **Authentication Key**.
|
||||||
You should see: Hi mrhavens! You’ve successfully authenticated....
|
* Key: Paste the public key.
|
||||||
4.2 Add SSH Key to Forgejo
|
* Click **Add SSH key**.
|
||||||
Copy the public key:
|
* Test the connection:
|
||||||
bash
|
* bash
|
||||||
cat ~/.ssh/forgejo_key.pub
|
* ssh \-i \~/.ssh/github\_key \-T git@github.com
|
||||||
Access Forgejo at http://localhost:3000 (ensure Forgejo is running via fold-stack).
|
* You should see: Hi mrhavens\! You’ve successfully authenticated....
|
||||||
Go to Settings > SSH / GPG Keys > Add Key.
|
|
||||||
Name: fold-stack-git-sync.
|
**4.2 Add SSH Key to Forgejo**
|
||||||
Content: Paste the public key.
|
|
||||||
Click Add Key.
|
* Copy the public key:
|
||||||
Test the connection:
|
* bash
|
||||||
bash
|
* cat \~/.ssh/forgejo\_key.pub
|
||||||
ssh -i ~/.ssh/forgejo_key -p 2222 -T git@localhost
|
* Access Forgejo at http://localhost:3000 (ensure Forgejo is running via fold-stack).
|
||||||
You should see a success message.
|
* Go to **Settings** \> **SSH / GPG Keys** \> **Add Key**.
|
||||||
5. Configure Rclone for Internet Archive and Web3.storage
|
* Name: fold-stack-git-sync.
|
||||||
|
* Content: Paste the public key.
|
||||||
|
* Click **Add Key**.
|
||||||
|
* Test the connection:
|
||||||
|
* bash
|
||||||
|
* ssh \-i \~/.ssh/forgejo\_key \-p 2222 \-T git@localhost
|
||||||
|
* You should see a success message.
|
||||||
|
|
||||||
|
**5\. Configure Rclone for Internet Archive and Web3.storage**
|
||||||
|
|
||||||
Git-Sync uses Rclone to sync Git bundles to Internet Archive and Web3.storage.
|
Git-Sync uses Rclone to sync Git bundles to Internet Archive and Web3.storage.
|
||||||
Run the Rclone configuration wizard:
|
|
||||||
bash
|
* Run the Rclone configuration wizard:
|
||||||
rclone config
|
* bash
|
||||||
Add the following remotes:
|
* rclone config
|
||||||
Internet Archive (ia):
|
* Add the following remotes:
|
||||||
Choose n (new remote).
|
* **Internet Archive (**ia**)**:
|
||||||
Name: ia
|
* Choose n (new remote).
|
||||||
Type: internetarchive.
|
* Name: ia
|
||||||
Access Key ID: Your Internet Archive access key (from archive.org account settings).
|
* Type: internetarchive.
|
||||||
Secret Access Key: Your Internet Archive secret key.
|
* Access Key ID: Your Internet Archive access key (from archive.org account settings).
|
||||||
Edit Advanced Config: n.
|
* Secret Access Key: Your Internet Archive secret key.
|
||||||
Web3.storage (web3):
|
* Edit Advanced Config: n.
|
||||||
Choose n (new remote).
|
* **Web3.storage (**web3**)**:
|
||||||
Name: web3
|
* Choose n (new remote).
|
||||||
Type: ipfs.
|
* Name: web3
|
||||||
Host: api.web3.storage.
|
* Type: ipfs.
|
||||||
API Token: Your Web3.storage API token (from web3.storage).
|
* Host: api.web3.storage.
|
||||||
Edit Advanced Config: n.
|
* API Token: Your Web3.storage API token (from web3.storage).
|
||||||
Copy the Rclone configuration to the project:
|
* Edit Advanced Config: n.
|
||||||
bash
|
* Copy the Rclone configuration to the project:
|
||||||
mkdir -p config/rclone
|
* bash
|
||||||
cp ~/.config/rclone/rclone.conf config/rclone/rclone.conf
|
|
||||||
chmod 600 config/rclone/rclone.conf
|
mkdir \-p config/rclone
|
||||||
Copy to git-sync config:
|
cp \~/.config/rclone/rclone.conf config/rclone/rclone.conf
|
||||||
bash
|
|
||||||
mkdir -p config/git-sync
|
* chmod 600 config/rclone/rclone.conf
|
||||||
cp config/rclone/rclone.conf config/git-sync/rclone.conf
|
* Copy to git-sync config:
|
||||||
6. Copy SSH Keys to git-sync Configuration
|
* bash
|
||||||
|
|
||||||
|
mkdir \-p config/git-sync
|
||||||
|
|
||||||
|
* cp config/rclone/rclone.conf config/git-sync/rclone.conf
|
||||||
|
|
||||||
|
**6\. Copy SSH Keys to** git-sync **Configuration**
|
||||||
|
|
||||||
Copy the private keys to the git-sync secrets directory:
|
Copy the private keys to the git-sync secrets directory:
|
||||||
|
|
||||||
bash
|
bash
|
||||||
mkdir -p config/git-sync/secrets
|
|
||||||
cp ~/.ssh/github_key config/git-sync/secrets/github.key
|
mkdir \-p config/git-sync/secrets
|
||||||
cp ~/.ssh/forgejo_key config/git-sync/secrets/forgejo.key
|
cp \~/.ssh/github\_key config/git-sync/secrets/github.key
|
||||||
|
cp \~/.ssh/forgejo\_key config/git-sync/secrets/forgejo.key
|
||||||
chmod 600 config/git-sync/secrets/github.key config/git-sync/secrets/forgejo.key
|
chmod 600 config/git-sync/secrets/github.key config/git-sync/secrets/forgejo.key
|
||||||
7. Configure remotes.conf
|
|
||||||
|
**7\. Configure** remotes.conf
|
||||||
|
|
||||||
Edit config/git-sync/remotes.conf to specify the remotes to sync to. Example:
|
Edit config/git-sync/remotes.conf to specify the remotes to sync to. Example:
|
||||||
|
|
||||||
github|git|git@github.com:mrhavens/mirror-repo.git|1
|
github|git|git@github.com:mrhavens/mirror-repo.git|1
|
||||||
forgejo|git|git@localhost:2222/mrhavens/mirror-repo.git|1
|
forgejo|git|git@localhost:2222/mrhavens/mirror-repo.git|1
|
||||||
radicle|radicle|radicle://mrhavens/mirror-repo|1
|
radicle|radicle|radicle://mrhavens/mirror-repo|1
|
||||||
ia|rclone|ia:fold-stack-git-mirror|1
|
ia|rclone|ia:fold-stack-git-mirror|1
|
||||||
web3|rclone|web3:fold-stack-git-mirror|0
|
web3|rclone|web3:fold-stack-git-mirror|0
|
||||||
Format: remote_name|type|url|enabled (1 for enabled, 0 for disabled).
|
|
||||||
Example Explanation:
|
* **Format**: remote\_name|type|url|enabled (1 for enabled, 0 for disabled).
|
||||||
github: Syncs to mrhavens/mirror-repo on GitHub via SSH.
|
* **Example Explanation**:
|
||||||
forgejo: Syncs to mrhavens/mirror-repo on your local Forgejo instance (port 2222).
|
* github: Syncs to mrhavens/mirror-repo on GitHub via SSH.
|
||||||
radicle: Placeholder for Radicle (not implemented).
|
* forgejo: Syncs to mrhavens/mirror-repo on your local Forgejo instance (port 2222).
|
||||||
ia: Syncs Git bundles to fold-stack-git-mirror on Internet Archive via Rclone.
|
* radicle: Placeholder for Radicle (not implemented).
|
||||||
web3: Syncs Git bundles to fold-stack-git-mirror on Web3.storage (disabled by default).
|
* ia: Syncs Git bundles to fold-stack-git-mirror on Internet Archive via Rclone.
|
||||||
8. Configure Push Rules (rules.json)
|
* web3: Syncs Git bundles to fold-stack-git-mirror on Web3.storage (disabled by default).
|
||||||
|
|
||||||
|
**8\. Configure Push Rules (**rules.json**)**
|
||||||
|
|
||||||
Edit config/git-sync/rules.json to define which branches to sync:
|
Edit config/git-sync/rules.json to define which branches to sync:
|
||||||
|
|
||||||
json
|
json
|
||||||
|
|
||||||
{
|
{
|
||||||
"branches": ["main", "dev"],
|
"branches": \["main", "dev"\],
|
||||||
"exclude_tags": ["v*"]
|
"exclude\_tags": \["v\*"\]
|
||||||
}
|
}
|
||||||
Example Explanation:
|
|
||||||
Syncs only the main and dev branches.
|
* **Example Explanation**:
|
||||||
Excludes tags starting with v (e.g., v1.0).
|
* Syncs only the main and dev branches.
|
||||||
Note: The current implementation syncs all branches (branches: ["*"]) and doesn’t exclude tags. Update entrypoint.sh to enforce these rules if needed.
|
* Excludes tags starting with v (e.g., v1.0).
|
||||||
9. Configure Environment Variables (.env)
|
|
||||||
|
**Note**: The current implementation syncs all branches (branches: \["\*"\]) and doesn’t exclude tags. Update entrypoint.sh to enforce these rules if needed.
|
||||||
|
|
||||||
|
**9\. Configure Environment Variables (**.env**)**
|
||||||
|
|
||||||
Edit config/git-sync/.env to set runtime options:
|
Edit config/git-sync/.env to set runtime options:
|
||||||
SYNC_INTERVAL=300
|
|
||||||
PUSH_MODE=push
|
SYNC\_INTERVAL=300
|
||||||
SIGN_COMMITS=false
|
PUSH\_MODE=push
|
||||||
LOG_LEVEL=INFO
|
SIGN\_COMMITS=false
|
||||||
RETRY_MAX=3
|
LOG\_LEVEL=INFO
|
||||||
RETRY_BACKOFF=5
|
RETRY\_MAX=3
|
||||||
Example Explanation:
|
RETRY\_BACKOFF=5
|
||||||
SYNC_INTERVAL=300: Check for changes every 300 seconds (5 minutes).
|
|
||||||
PUSH_MODE=push: Use git push for Git remotes (alternative: bundle for Git bundles).
|
* **Example Explanation**:
|
||||||
SIGN_COMMITS=false: Disable GPG commit signing (placeholder).
|
* SYNC\_INTERVAL=300: Check for changes every 300 seconds (5 minutes).
|
||||||
LOG_LEVEL=INFO: Log all messages (alternative: ERROR for errors only).
|
* PUSH\_MODE=push: Use git push for Git remotes (alternative: bundle for Git bundles).
|
||||||
RETRY_MAX=3: Retry failed syncs up to 3 times.
|
* SIGN\_COMMITS=false: Disable GPG commit signing (placeholder).
|
||||||
RETRY_BACKOFF=5: Wait 5 seconds (exponential increase) between retries.
|
* LOG\_LEVEL=INFO: Log all messages (alternative: ERROR for errors only).
|
||||||
10. Start the git-sync Service
|
* RETRY\_MAX=3: Retry failed syncs up to 3 times.
|
||||||
|
* RETRY\_BACKOFF=5: Wait 5 seconds (exponential increase) between retries.
|
||||||
|
|
||||||
|
**10\. Start the** git-sync **Service**
|
||||||
|
|
||||||
Git-Sync is integrated into fold-stack’s docker-compose.dev.yml. Start the service:
|
Git-Sync is integrated into fold-stack’s docker-compose.dev.yml. Start the service:
|
||||||
|
|
||||||
bash
|
bash
|
||||||
|
|
||||||
cd fold-stack
|
cd fold-stack
|
||||||
./scripts/up-dev.sh
|
./scripts/up-dev.sh
|
||||||
|
|
||||||
This starts all fold-stack services, including git-sync.
|
This starts all fold-stack services, including git-sync.
|
||||||
🌐 Usage
|
|
||||||
1. Add a Commit to the Local Repository
|
|
||||||
Make changes to the local repository and commit them:
|
|
||||||
bash
|
|
||||||
cd volumes/repos
|
|
||||||
echo "New feature" >> README.md
|
|
||||||
git add .
|
|
||||||
git commit -m "Added new feature"
|
|
||||||
2. Wait for Automated Sync
|
|
||||||
Git-Sync will detect changes within SYNC_INTERVAL (default: 300 seconds) and sync to all enabled remotes. Monitor the logs:
|
|
||||||
bash
|
|
||||||
docker logs git_sync_dev --follow
|
|
||||||
3. Manually Trigger a Sync
|
|
||||||
To sync immediately, use the manual push script:
|
|
||||||
bash
|
|
||||||
./scripts/manual-push-git-sync.sh
|
|
||||||
4. Verify Sync
|
|
||||||
GitHub: Check https://github.com/mrhavens/mirror-repo.
|
|
||||||
Forgejo: Check http://localhost:3000/mrhavens/mirror-repo.
|
|
||||||
Internet Archive: Check fold-stack-git-mirror on archive.org.
|
|
||||||
Web3.storage: Enable in remotes.conf and check fold-stack-git-mirror.
|
|
||||||
5. Generate a Sync Report
|
|
||||||
View the latest sync activity for each remote:
|
|
||||||
bash
|
|
||||||
./scripts/report-git-sync.sh
|
|
||||||
Example Output:
|
|
||||||
```
|
|
||||||
📊 GIT-SYNC SYNC REPORT
|
|
||||||
📅 Date: Mon May 26 22:41:00 CDT 2025
|
|
||||||
📌 Local Repository Latest Commit
|
|
||||||
Commit: a1b2c3d4e5f6g7h8i9j0k1l2m3n4o5p6
|
|
||||||
Message: Added new feature
|
|
||||||
Time: Mon May 26 22:40:00 CDT 2025
|
|
||||||
📌 Latest Sync Activity by Remote
|
|
||||||
Remote: github (git, git@github.com:mrhavens/mirror-repo.git)
|
|
||||||
Last Synced Commit: a1b2c3d4e5f6g7h8i9j0k1l2m3n4o5p6
|
|
||||||
Commit Message: Added new feature
|
|
||||||
Timestamp: [Mon May 26 22:41:00 CDT 2025]
|
|
||||||
✅ Status: Successfully synced
|
|
||||||
Remote: forgejo (git, git@localhost:2222/mrhavens/mirror-repo.git)
|
|
||||||
Last Synced Commit: a1b2c3d4e5f6g7h8i9j0k1l2m3n4o5p6
|
|
||||||
Commit Message: Added new feature
|
|
||||||
Timestamp: [Mon May 26 22:41:01 CDT 2025]
|
|
||||||
✅ Status: Successfully synced
|
|
||||||
Remote: ia (rclone, ia:fold-stack-git-mirror)
|
|
||||||
Last Synced Bundle: repo-1716777660.bundle
|
|
||||||
Timestamp: [Mon May 26 22:41:02 CDT 2025]
|
|
||||||
✅ Status: Successfully synced
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 🛠️ Troubleshooting
|
🌐 **Usage**
|
||||||
|
|
||||||
### 1. Run Diagnostics
|
**1\. Add a Commit to the Local Repository**
|
||||||
|
|
||||||
|
Make changes to the local repository and commit them:
|
||||||
|
|
||||||
|
bash
|
||||||
|
|
||||||
|
cd volumes/repos
|
||||||
|
echo "New feature" \>\> README.md
|
||||||
|
git add .
|
||||||
|
git commit \-m "Added new feature"
|
||||||
|
|
||||||
|
**2\. Wait for Automated Sync**
|
||||||
|
|
||||||
|
Git-Sync will detect changes within SYNC\_INTERVAL (default: 300 seconds) and sync to all enabled remotes. Monitor the logs:
|
||||||
|
|
||||||
|
bash
|
||||||
|
|
||||||
|
docker logs git\_sync\_dev \--follow
|
||||||
|
|
||||||
|
**3\. Manually Trigger a Sync**
|
||||||
|
|
||||||
|
To sync immediately, use the manual push script:
|
||||||
|
|
||||||
|
bash
|
||||||
|
|
||||||
|
./scripts/manual-push-git-sync.sh
|
||||||
|
|
||||||
|
**4\. Verify Sync**
|
||||||
|
|
||||||
|
* **GitHub**: Check https://github.com/mrhavens/mirror-repo.
|
||||||
|
* **Forgejo**: Check http://localhost:3000/mrhavens/mirror-repo.
|
||||||
|
* **Internet Archive**: Check fold-stack-git-mirror on archive.org.
|
||||||
|
* **Web3.storage**: Enable in remotes.conf and check fold-stack-git-mirror.
|
||||||
|
|
||||||
|
**5\. Generate a Sync Report**
|
||||||
|
|
||||||
|
View the latest sync activity for each remote:
|
||||||
|
|
||||||
|
bash
|
||||||
|
|
||||||
|
./scripts/report-git-sync.sh
|
||||||
|
|
||||||
|
**Example Output: \`\`\`**
|
||||||
|
|
||||||
|
**📊 GIT-SYNC SYNC REPORT**
|
||||||
|
|
||||||
|
**📅** Date: Mon May 26 22:41:00 CDT 2025
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
📌 **Local Repository Latest Commit**
|
||||||
|
|
||||||
|
Commit: a1b2c3d4e5f6g7h8i9j0k1l2m3n4o5p6 Message: Added new feature Time: Mon May 26 22:40:00 CDT 2025
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
📌 **Latest Sync Activity by Remote**
|
||||||
|
|
||||||
|
Remote: github (git, git@github.com:mrhavens/mirror-repo.git) Last Synced Commit: a1b2c3d4e5f6g7h8i9j0k1l2m3n4o5p6 Commit Message: Added new feature Timestamp: \[Mon May 26 22:41:00 CDT 2025\] ✅ Status: Successfully synced
|
||||||
|
|
||||||
|
Remote: forgejo (git, git@localhost:2222/mrhavens/mirror-repo.git) Last Synced Commit: a1b2c3d4e5f6g7h8i9j0k1l2m3n4o5p6 Commit Message: Added new feature Timestamp: \[Mon May 26 22:41:01 CDT 2025\] ✅ Status: Successfully synced
|
||||||
|
|
||||||
|
Remote: ia (rclone, ia:fold-stack-git-mirror) Last Synced Bundle: repo-1716777660.bundle Timestamp: \[Mon May 26 22:41:02 CDT 2025\] ✅ Status: Successfully synced
|
||||||
|
|
||||||
|
\---
|
||||||
|
|
||||||
|
\#\# 🛠️ Troubleshooting
|
||||||
|
|
||||||
|
\#\#\# 1\. Run Diagnostics
|
||||||
|
|
||||||
If sync fails, run the diagnostic script:
|
If sync fails, run the diagnostic script:
|
||||||
|
|
||||||
```bash
|
\`\`\`bash
|
||||||
./scripts/diagnose-git-sync.sh
|
./scripts/diagnose-git-sync.sh
|
||||||
Example Output:
|
|
||||||
```
|
|
||||||
🩺 GIT-SYNC COMPREHENSIVE DIAGNOSTICS
|
|
||||||
📅 Date: Mon May 26 22:41:00 CDT 2025
|
|
||||||
📌 SSH Keys Check
|
|
||||||
✅ /config/git-sync/secrets/github.key exists.
|
|
||||||
✅ /config/git-sync/secrets/github.key has correct permissions (600).
|
|
||||||
✅ /config/git-sync/secrets/forgejo.key exists.
|
|
||||||
✅ /config/git-sync/secrets/forgejo.key has correct permissions (600).
|
|
||||||
📌 Remote Connectivity Test
|
|
||||||
Testing github (git)...
|
|
||||||
✅ github connectivity test passed.
|
|
||||||
Testing forgejo (git)...
|
|
||||||
✅ forgejo connectivity test passed.
|
|
||||||
Testing ia (rclone)...
|
|
||||||
❌ ia connectivity test failed. Check rclone.conf or credentials.
|
|
||||||
|
|
||||||
### 2. Common Issues and Fixes
|
**Example Output: \`\`\`**
|
||||||
|
|
||||||
- **Container Not Running**:
|
**🩺 GIT-SYNC COMPREHENSIVE DIAGNOSTICS**
|
||||||
```bash
|
|
||||||
|
**📅** Date: Mon May 26 22:41:00 CDT 2025
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
📌 **SSH Keys Check**
|
||||||
|
|
||||||
|
**✅** /config/git-sync/secrets/github.key exists. ✅ /config/git-sync/secrets/github.key has correct permissions (600). ✅ /config/git-sync/secrets/forgejo.key exists. ✅ /config/git-sync/secrets/forgejo.key has correct permissions (600).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
📌 **Remote Connectivity Test**
|
||||||
|
|
||||||
|
Testing github (git)... ✅ github connectivity test passed. Testing forgejo (git)... ✅ forgejo connectivity test passed. Testing ia (rclone)... ❌ ia connectivity test failed. Check rclone.conf or credentials.
|
||||||
|
|
||||||
|
\#\#\# 2\. Common Issues and Fixes
|
||||||
|
|
||||||
|
\- \*\*Container Not Running\*\*:
|
||||||
|
\`\`\`bash
|
||||||
./scripts/down-dev.sh && ./scripts/up-dev.sh
|
./scripts/down-dev.sh && ./scripts/up-dev.sh
|
||||||
SSH Key Issues:
|
|
||||||
Verify permissions: chmod 600 config/git-sync/secrets/*.
|
* **SSH Key Issues**:
|
||||||
Test connectivity: ssh -i ~/.ssh/github_key -T git@github.com.
|
* Verify permissions: chmod 600 config/git-sync/secrets/\*.
|
||||||
Rclone Remote Fails:
|
* Test connectivity: ssh \-i \~/.ssh/github\_key \-T git@github.com.
|
||||||
Reconfigure Rclone: rclone config.
|
* **Rclone Remote Fails**:
|
||||||
Verify remotes: rclone listremotes --config config/git-sync/rclone.conf.
|
* Reconfigure Rclone: rclone config.
|
||||||
Logs Missing:
|
* Verify remotes: rclone listremotes \--config config/git-sync/rclone.conf.
|
||||||
Check volume permissions: chmod -R 775 volumes/logs && chown -R 1000:1000 volumes/logs.
|
* **Logs Missing**:
|
||||||
📚 Advanced Configuration
|
* Check volume permissions: chmod \-R 775 volumes/logs && chown \-R 1000:1000 volumes/logs.
|
||||||
1. Enable Web3.storage Sync
|
|
||||||
Edit config/git-sync/remotes.conf to enable Web3.storage:
|
---
|
||||||
web3|rclone|web3:fold-stack-git-mirror|1
|
|
||||||
2. Add a New Remote (e.g., S3)
|
📚 **Advanced Configuration**
|
||||||
Add a new remote to remotes.conf using Rclone:
|
|
||||||
s3|rclone|s3:fold-stack-git-mirror|1
|
**1\. Enable Web3.storage Sync**
|
||||||
Configure the s3 remote in config/git-sync/rclone.conf using rclone config.
|
|
||||||
3. Adjust Sync Interval
|
Edit config/git-sync/remotes.conf to enable Web3.storage:
|
||||||
Edit config/git-sync/.env to change the sync interval:
|
|
||||||
SYNC_INTERVAL=60 # Check every 60 seconds
|
web3|rclone|web3:fold-stack-git-mirror|1
|
||||||
Restart the service:
|
|
||||||
bash
|
**2\. Add a New Remote (e.g., S3)**
|
||||||
docker compose -f docker-compose.dev.yml stop git-sync
|
|
||||||
docker compose -f docker-compose.dev.yml up -d git-sync
|
Add a new remote to remotes.conf using Rclone:
|
||||||
4. Enable Commit Signing (Future Feature)
|
|
||||||
To enable GPG commit signing (not yet implemented), set:
|
s3|rclone|s3:fold-stack-git-mirror|1
|
||||||
SIGN_COMMITS=true
|
|
||||||
You’ll need to:
|
Configure the s3 remote in config/git-sync/rclone.conf using rclone config.
|
||||||
Add GPG keys to the container.
|
|
||||||
Update entrypoint.sh to sign commits using git.
|
**3\. Adjust Sync Interval**
|
||||||
📅 Last Updated
|
|
||||||
This README was last updated on May 26, 2025, at 10:54 PM CDT.
|
Edit config/git-sync/.env to change the sync interval:
|
||||||
|
|
||||||
|
SYNC\_INTERVAL=60 \# Check every 60 seconds
|
||||||
|
|
||||||
|
Restart the service:
|
||||||
|
|
||||||
|
bash
|
||||||
|
|
||||||
|
docker compose \-f docker-compose.dev.yml stop git-sync
|
||||||
|
docker compose \-f docker-compose.dev.yml up \-d git-sync
|
||||||
|
|
||||||
|
**4\. Enable Commit Signing (Future Feature)**
|
||||||
|
|
||||||
|
To enable GPG commit signing (not yet implemented), set:
|
||||||
|
|
||||||
|
SIGN\_COMMITS=true
|
||||||
|
|
||||||
|
You’ll need to:
|
||||||
|
|
||||||
|
* Add GPG keys to the container.
|
||||||
|
* Update entrypoint.sh to sign commits using git.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
📅 **Last Updated**
|
||||||
|
|
||||||
|
This README was last updated on **May 26, 2025, at 10:54 PM CDT**.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user