updated readme
This commit is contained in:
+170
-318
@@ -1,62 +1,40 @@
|
|||||||
# Git-Sync Mirror Agent
|
Git-Sync Mirror Agent
|
||||||
|
Git-Sync is a containerized mirroring agent designed to watch a local Git repository and propagate changes to multiple remote repositories with high reliability, auditability, and fault recovery. It supports mirroring to platforms like GitHub, Forgejo, Radicle, Internet Archive, and Web3.storage, ensuring your repository is resiliently backed up across diverse infrastructure. Git-Sync is a key component of the fold-stack project, emphasizing sovereignty and fieldcraft-resilient design.
|
||||||
**Git-Sync** is a containerized mirroring agent designed to watch a local Git repository and propagate changes to multiple remote repositories with high reliability, auditability, and fault recovery. It supports mirroring to platforms like GitHub, Forgejo, Radicle, Internet Archive, and Web3.storage, ensuring your repository is resiliently backed up across diverse infrastructure. Git-Sync is a key component of the `fold-stack` project, emphasizing sovereignty and fieldcraft-resilient design.
|
📜 Overview
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 📜 Overview
|
|
||||||
|
|
||||||
Git-Sync operates as a Docker container that:
|
Git-Sync operates as a Docker container that:
|
||||||
|
Watches a local Git repository for changes.
|
||||||
- Watches a local Git repository for changes.
|
Syncs changes to multiple configured remotes (GitHub, Forgejo, Radicle, Internet Archive, Web3.storage).
|
||||||
- Syncs changes to multiple configured remotes (GitHub, Forgejo, Radicle, Internet Archive, Web3.storage).
|
Ensures atomicity using lockfiles.
|
||||||
- Ensures atomicity using lockfiles.
|
Logs all operations for auditability.
|
||||||
- Logs all operations for auditability.
|
Implements failure recovery with retries and exponential backoff.
|
||||||
- Implements failure recovery with retries and exponential backoff.
|
Operates independently of commercial SaaS infrastructure.
|
||||||
- Operates independently of commercial SaaS infrastructure.
|
Supported Remotes
|
||||||
|
GitHub: Push via SSH.
|
||||||
### Supported Remotes
|
Forgejo: Push via SSH to a self-hosted instance.
|
||||||
- **GitHub**: Push via SSH.
|
Radicle: Peer-to-peer Git (placeholder, requires rad CLI implementation).
|
||||||
- **Forgejo**: Push via SSH to a self-hosted instance.
|
Internet Archive: Git bundles uploaded via Rclone.
|
||||||
- **Radicle**: Peer-to-peer Git (placeholder, requires `rad` CLI implementation).
|
Web3.storage: Git bundles uploaded via Rclone (optional).
|
||||||
- **Internet Archive**: Git bundles uploaded via Rclone.
|
Extendable: Can be extended to support S3, IPFS, Sia, etc.
|
||||||
- **Web3.storage**: Git bundles uploaded via Rclone (optional).
|
🛠️ Prerequisites
|
||||||
- **Extendable**: Can be extended to support S3, IPFS, Sia, etc.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🛠️ Prerequisites
|
|
||||||
|
|
||||||
Before setting up Git-Sync, ensure you have the following:
|
Before setting up Git-Sync, ensure you have the following:
|
||||||
|
Docker and Docker Compose installed.
|
||||||
- **Docker** and **Docker Compose** installed.
|
Install Docker: Official Docker Installation Guide
|
||||||
- Install Docker: [Official Docker Installation Guide](https://docs.docker.com/get-docker/)
|
Install Docker Compose: Official Docker Compose Installation Guide
|
||||||
- Install Docker Compose: [Official Docker Compose Installation Guide](https://docs.docker.com/compose/install/)
|
Git installed for managing the local repository.
|
||||||
- **Git** installed for managing the local repository.
|
Install Git: Official Git Installation Guide
|
||||||
- Install Git: [Official Git Installation Guide](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git)
|
A local Git repository to mirror (e.g., volumes/repos in fold-stack).
|
||||||
- A local Git repository to mirror (e.g., `volumes/repos` in `fold-stack`).
|
SSH keys for GitHub and Forgejo.
|
||||||
- SSH keys for GitHub and Forgejo.
|
Rclone configured for Internet Archive and Web3.storage (if using these remotes).
|
||||||
- Rclone configured for Internet Archive and Web3.storage (if using these remotes).
|
A machine with at least 2GB of RAM (Git-Sync is lightweight but depends on fold-stack requirements).
|
||||||
- A machine with at least 2GB of RAM (Git-Sync is lightweight but depends on `fold-stack` requirements).
|
🚀 Setup Instructions
|
||||||
|
1. Clone the fold-stack Repository
|
||||||
---
|
Git-Sync is part of the fold-stack project. Clone the repository if you haven’t already:
|
||||||
|
bash
|
||||||
## 🚀 Setup Instructions
|
|
||||||
|
|
||||||
### 1. Clone the `fold-stack` Repository
|
|
||||||
|
|
||||||
Git-Sync is part of the `fold-stack` project. Clone the repository if you haven’t already:
|
|
||||||
|
|
||||||
\`\`\`bash
|
|
||||||
git clone https://github.com/mrhavens/fold-stack.git
|
git clone https://github.com/mrhavens/fold-stack.git
|
||||||
cd fold-stack
|
cd fold-stack
|
||||||
\`\`\`
|
2. Initialize a Local Git Repository
|
||||||
|
Git-Sync watches a local repository at volumes/repos. Initialize it if it doesn’t exist:
|
||||||
### 2. Initialize a Local Git Repository
|
bash
|
||||||
|
|
||||||
Git-Sync watches a local repository at `volumes/repos`. Initialize it if it doesn’t exist:
|
|
||||||
|
|
||||||
\`\`\`bash
|
|
||||||
mkdir -p volumes/repos
|
mkdir -p volumes/repos
|
||||||
cd volumes/repos
|
cd volumes/repos
|
||||||
git init
|
git init
|
||||||
@@ -65,254 +43,173 @@ git add .
|
|||||||
git commit -m "Initial commit"
|
git commit -m "Initial commit"
|
||||||
git branch -M main
|
git branch -M main
|
||||||
cd ../..
|
cd ../..
|
||||||
\`\`\`
|
3. Generate SSH Keys for GitHub and Forgejo
|
||||||
|
|
||||||
### 3. Generate SSH Keys for GitHub and Forgejo
|
|
||||||
|
|
||||||
Git-Sync uses SSH keys to authenticate with GitHub and Forgejo.
|
Git-Sync uses SSH keys to authenticate with GitHub and Forgejo.
|
||||||
|
3.1 Generate SSH Key for GitHub
|
||||||
#### 3.1 Generate SSH Key for GitHub
|
bash
|
||||||
\`\`\`bash
|
|
||||||
ssh-keygen -t ed25519 -C "your_email@example.com" -f ~/.ssh/github_key
|
ssh-keygen -t ed25519 -C "your_email@example.com" -f ~/.ssh/github_key
|
||||||
\`\`\`
|
3.2 Generate SSH Key for Forgejo
|
||||||
|
bash
|
||||||
#### 3.2 Generate SSH Key for Forgejo
|
|
||||||
\`\`\`bash
|
|
||||||
ssh-keygen -t ed25519 -C "your_email@example.com" -f ~/.ssh/forgejo_key
|
ssh-keygen -t ed25519 -C "your_email@example.com" -f ~/.ssh/forgejo_key
|
||||||
\`\`\`
|
Press Enter to skip setting a passphrase, or set one for added security.
|
||||||
|
This creates ~/.ssh/github_key and ~/.ssh/forgejo_key (private keys) and their .pub counterparts (public keys).
|
||||||
- Press Enter to skip setting a passphrase, or set one for added security.
|
4. Configure SSH Keys on GitHub and Forgejo
|
||||||
- This creates `~/.ssh/github_key` and `~/.ssh/forgejo_key` (private keys) and their `.pub` counterparts (public keys).
|
4.1 Add SSH Key to GitHub
|
||||||
|
Copy the public key:
|
||||||
### 4. Configure SSH Keys on GitHub and Forgejo
|
bash
|
||||||
|
cat ~/.ssh/github_key.pub
|
||||||
#### 4.1 Add SSH Key to GitHub
|
Go to GitHub > Settings > SSH and GPG keys > New SSH key.
|
||||||
1. Copy the public key:
|
Title: fold-stack-git-sync.
|
||||||
\`\`\`bash
|
Key type: Authentication Key.
|
||||||
cat ~/.ssh/github_key.pub
|
Key: Paste the public key.
|
||||||
\`\`\`
|
Click Add SSH key.
|
||||||
2. Go to [GitHub](https://github.com) > **Settings** > **SSH and GPG keys** > **New SSH key**.
|
Test the connection:
|
||||||
3. Title: `fold-stack-git-sync`.
|
bash
|
||||||
4. Key type: **Authentication Key**.
|
ssh -i ~/.ssh/github_key -T git@github.com
|
||||||
5. Key: Paste the public key.
|
You should see: Hi mrhavens! You’ve successfully authenticated....
|
||||||
6. Click **Add SSH key**.
|
4.2 Add SSH Key to Forgejo
|
||||||
7. Test the connection:
|
Copy the public key:
|
||||||
\`\`\`bash
|
bash
|
||||||
ssh -i ~/.ssh/github_key -T git@github.com
|
cat ~/.ssh/forgejo_key.pub
|
||||||
\`\`\`
|
Access Forgejo at http://localhost:3000 (ensure Forgejo is running via fold-stack).
|
||||||
You should see: `Hi mrhavens! You’ve successfully authenticated...`.
|
Go to Settings > SSH / GPG Keys > Add Key.
|
||||||
|
Name: fold-stack-git-sync.
|
||||||
#### 4.2 Add SSH Key to Forgejo
|
Content: Paste the public key.
|
||||||
1. Copy the public key:
|
Click Add Key.
|
||||||
\`\`\`bash
|
Test the connection:
|
||||||
cat ~/.ssh/forgejo_key.pub
|
bash
|
||||||
\`\`\`
|
ssh -i ~/.ssh/forgejo_key -p 2222 -T git@localhost
|
||||||
2. Access Forgejo at `http://localhost:3000` (ensure Forgejo is running via `fold-stack`).
|
You should see a success message.
|
||||||
3. Go to **Settings** > **SSH / GPG Keys** > **Add Key**.
|
5. Configure Rclone for Internet Archive and Web3.storage
|
||||||
4. Name: `fold-stack-git-sync`.
|
|
||||||
5. Content: Paste the public key.
|
|
||||||
6. Click **Add Key**.
|
|
||||||
7. Test the connection:
|
|
||||||
\`\`\`bash
|
|
||||||
ssh -i ~/.ssh/forgejo_key -p 2222 -T git@localhost
|
|
||||||
\`\`\`
|
|
||||||
You should see a success message.
|
|
||||||
|
|
||||||
### 5. Configure Rclone for Internet Archive and Web3.storage
|
|
||||||
|
|
||||||
Git-Sync uses Rclone to sync Git bundles to Internet Archive and Web3.storage.
|
Git-Sync uses Rclone to sync Git bundles to Internet Archive and Web3.storage.
|
||||||
|
Run the Rclone configuration wizard:
|
||||||
1. Run the Rclone configuration wizard:
|
bash
|
||||||
\`\`\`bash
|
rclone config
|
||||||
rclone config
|
Add the following remotes:
|
||||||
\`\`\`
|
Internet Archive (ia):
|
||||||
2. Add the following remotes:
|
Choose n (new remote).
|
||||||
- **Internet Archive (`ia`)**:
|
Name: ia
|
||||||
- Choose `n` (new remote).
|
Type: internetarchive.
|
||||||
- Name: `ia`
|
Access Key ID: Your Internet Archive access key (from archive.org account settings).
|
||||||
- Type: `internetarchive`.
|
Secret Access Key: Your Internet Archive secret key.
|
||||||
- Access Key ID: Your Internet Archive access key (from archive.org account settings).
|
Edit Advanced Config: n.
|
||||||
- Secret Access Key: Your Internet Archive secret key.
|
Web3.storage (web3):
|
||||||
- Edit Advanced Config: `n`.
|
Choose n (new remote).
|
||||||
- **Web3.storage (`web3`)**:
|
Name: web3
|
||||||
- Choose `n` (new remote).
|
Type: ipfs.
|
||||||
- Name: `web3`
|
Host: api.web3.storage.
|
||||||
- Type: `ipfs`.
|
API Token: Your Web3.storage API token (from web3.storage).
|
||||||
- Host: `api.web3.storage`.
|
Edit Advanced Config: n.
|
||||||
- API Token: Your Web3.storage API token (from web3.storage).
|
Copy the Rclone configuration to the project:
|
||||||
- Edit Advanced Config: `n`.
|
bash
|
||||||
3. Copy the Rclone configuration to the project:
|
mkdir -p config/rclone
|
||||||
\`\`\`bash
|
cp ~/.config/rclone/rclone.conf config/rclone/rclone.conf
|
||||||
mkdir -p config/rclone
|
chmod 600 config/rclone/rclone.conf
|
||||||
cp ~/.config/rclone/rclone.conf config/rclone/rclone.conf
|
Copy to git-sync config:
|
||||||
chmod 600 config/rclone/rclone.conf
|
bash
|
||||||
\`\`\`
|
mkdir -p config/git-sync
|
||||||
4. Copy to `git-sync` config:
|
cp config/rclone/rclone.conf config/git-sync/rclone.conf
|
||||||
\`\`\`bash
|
6. Copy SSH Keys to git-sync Configuration
|
||||||
mkdir -p config/git-sync
|
Copy the private keys to the git-sync secrets directory:
|
||||||
cp config/rclone/rclone.conf config/git-sync/rclone.conf
|
bash
|
||||||
\`\`\`
|
|
||||||
|
|
||||||
### 6. Copy SSH Keys to `git-sync` Configuration
|
|
||||||
|
|
||||||
Copy the private keys to the `git-sync` secrets directory:
|
|
||||||
|
|
||||||
\`\`\`bash
|
|
||||||
mkdir -p config/git-sync/secrets
|
mkdir -p config/git-sync/secrets
|
||||||
cp ~/.ssh/github_key config/git-sync/secrets/github.key
|
cp ~/.ssh/github_key config/git-sync/secrets/github.key
|
||||||
cp ~/.ssh/forgejo_key config/git-sync/secrets/forgejo.key
|
cp ~/.ssh/forgejo_key config/git-sync/secrets/forgejo.key
|
||||||
chmod 600 config/git-sync/secrets/github.key config/git-sync/secrets/forgejo.key
|
chmod 600 config/git-sync/secrets/github.key config/git-sync/secrets/forgejo.key
|
||||||
\`\`\`
|
7. Configure remotes.conf
|
||||||
|
Edit config/git-sync/remotes.conf to specify the remotes to sync to. Example:
|
||||||
### 7. Configure `remotes.conf`
|
|
||||||
|
|
||||||
Edit `config/git-sync/remotes.conf` to specify the remotes to sync to. Example:
|
|
||||||
|
|
||||||
\`\`\`
|
|
||||||
github|git|git@github.com:mrhavens/mirror-repo.git|1
|
github|git|git@github.com:mrhavens/mirror-repo.git|1
|
||||||
forgejo|git|git@localhost:2222/mrhavens/mirror-repo.git|1
|
forgejo|git|git@localhost:2222/mrhavens/mirror-repo.git|1
|
||||||
radicle|radicle|radicle://mrhavens/mirror-repo|1
|
radicle|radicle|radicle://mrhavens/mirror-repo|1
|
||||||
ia|rclone|ia:fold-stack-git-mirror|1
|
ia|rclone|ia:fold-stack-git-mirror|1
|
||||||
web3|rclone|web3:fold-stack-git-mirror|0
|
web3|rclone|web3:fold-stack-git-mirror|0
|
||||||
\`\`\`
|
Format: remote_name|type|url|enabled (1 for enabled, 0 for disabled).
|
||||||
|
Example Explanation:
|
||||||
- **Format**: `remote_name|type|url|enabled` (1 for enabled, 0 for disabled).
|
github: Syncs to mrhavens/mirror-repo on GitHub via SSH.
|
||||||
- **Example Explanation**:
|
forgejo: Syncs to mrhavens/mirror-repo on your local Forgejo instance (port 2222).
|
||||||
- `github`: Syncs to `mrhavens/mirror-repo` on GitHub via SSH.
|
radicle: Placeholder for Radicle (not implemented).
|
||||||
- `forgejo`: Syncs to `mrhavens/mirror-repo` on your local Forgejo instance (port 2222).
|
ia: Syncs Git bundles to fold-stack-git-mirror on Internet Archive via Rclone.
|
||||||
- `radicle`: Placeholder for Radicle (not implemented).
|
web3: Syncs Git bundles to fold-stack-git-mirror on Web3.storage (disabled by default).
|
||||||
- `ia`: Syncs Git bundles to `fold-stack-git-mirror` on Internet Archive via Rclone.
|
8. Configure Push Rules (rules.json)
|
||||||
- `web3`: Syncs Git bundles to `fold-stack-git-mirror` on Web3.storage (disabled by default).
|
Edit config/git-sync/rules.json to define which branches to sync:
|
||||||
|
json
|
||||||
### 8. Configure Push Rules (`rules.json`)
|
|
||||||
|
|
||||||
Edit `config/git-sync/rules.json` to define which branches to sync:
|
|
||||||
|
|
||||||
\`\`\`json
|
|
||||||
{
|
{
|
||||||
"branches": ["main", "dev"],
|
"branches": ["main", "dev"],
|
||||||
"exclude_tags": ["v*"]
|
"exclude_tags": ["v*"]
|
||||||
}
|
}
|
||||||
\`\`\`
|
Example Explanation:
|
||||||
|
Syncs only the main and dev branches.
|
||||||
- **Example Explanation**:
|
Excludes tags starting with v (e.g., v1.0).
|
||||||
- Syncs only the `main` and `dev` branches.
|
Note: The current implementation syncs all branches (branches: ["*"]) and doesn’t exclude tags. Update entrypoint.sh to enforce these rules if needed.
|
||||||
- Excludes tags starting with `v` (e.g., `v1.0`).
|
9. Configure Environment Variables (.env)
|
||||||
|
Edit config/git-sync/.env to set runtime options:
|
||||||
**Note**: The current implementation syncs all branches (`branches: ["*"]`) and doesn’t exclude tags. Update `entrypoint.sh` to enforce these rules if needed.
|
|
||||||
|
|
||||||
### 9. Configure Environment Variables (`.env`)
|
|
||||||
|
|
||||||
Edit `config/git-sync/.env` to set runtime options:
|
|
||||||
|
|
||||||
\`\`\`
|
|
||||||
SYNC_INTERVAL=300
|
SYNC_INTERVAL=300
|
||||||
PUSH_MODE=push
|
PUSH_MODE=push
|
||||||
SIGN_COMMITS=false
|
SIGN_COMMITS=false
|
||||||
LOG_LEVEL=INFO
|
LOG_LEVEL=INFO
|
||||||
RETRY_MAX=3
|
RETRY_MAX=3
|
||||||
RETRY_BACKOFF=5
|
RETRY_BACKOFF=5
|
||||||
\`\`\`
|
Example Explanation:
|
||||||
|
SYNC_INTERVAL=300: Check for changes every 300 seconds (5 minutes).
|
||||||
- **Example Explanation**:
|
PUSH_MODE=push: Use git push for Git remotes (alternative: bundle for Git bundles).
|
||||||
- `SYNC_INTERVAL=300`: Check for changes every 300 seconds (5 minutes).
|
SIGN_COMMITS=false: Disable GPG commit signing (placeholder).
|
||||||
- `PUSH_MODE=push`: Use `git push` for Git remotes (alternative: `bundle` for Git bundles).
|
LOG_LEVEL=INFO: Log all messages (alternative: ERROR for errors only).
|
||||||
- `SIGN_COMMITS=false`: Disable GPG commit signing (placeholder).
|
RETRY_MAX=3: Retry failed syncs up to 3 times.
|
||||||
- `LOG_LEVEL=INFO`: Log all messages (alternative: `ERROR` for errors only).
|
RETRY_BACKOFF=5: Wait 5 seconds (exponential increase) between retries.
|
||||||
- `RETRY_MAX=3`: Retry failed syncs up to 3 times.
|
10. Start the git-sync Service
|
||||||
- `RETRY_BACKOFF=5`: Wait 5 seconds (exponential increase) between retries.
|
Git-Sync is integrated into fold-stack’s docker-compose.dev.yml. Start the service:
|
||||||
|
bash
|
||||||
### 10. Start the `git-sync` Service
|
|
||||||
|
|
||||||
Git-Sync is integrated into `fold-stack`’s `docker-compose.dev.yml`. Start the service:
|
|
||||||
|
|
||||||
\`\`\`bash
|
|
||||||
cd fold-stack
|
cd fold-stack
|
||||||
./scripts/up-dev.sh
|
./scripts/up-dev.sh
|
||||||
\`\`\`
|
This starts all fold-stack services, including git-sync.
|
||||||
|
🌐 Usage
|
||||||
This starts all `fold-stack` services, including `git-sync`.
|
1. Add a Commit to the Local Repository
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🌐 Usage
|
|
||||||
|
|
||||||
### 1. Add a Commit to the Local Repository
|
|
||||||
|
|
||||||
Make changes to the local repository and commit them:
|
Make changes to the local repository and commit them:
|
||||||
|
bash
|
||||||
\`\`\`bash
|
|
||||||
cd volumes/repos
|
cd volumes/repos
|
||||||
echo "New feature" >> README.md
|
echo "New feature" >> README.md
|
||||||
git add .
|
git add .
|
||||||
git commit -m "Added new feature"
|
git commit -m "Added new feature"
|
||||||
\`\`\`
|
2. Wait for Automated Sync
|
||||||
|
Git-Sync will detect changes within SYNC_INTERVAL (default: 300 seconds) and sync to all enabled remotes. Monitor the logs:
|
||||||
### 2. Wait for Automated Sync
|
bash
|
||||||
|
|
||||||
Git-Sync will detect changes within `SYNC_INTERVAL` (default: 300 seconds) and sync to all enabled remotes. Monitor the logs:
|
|
||||||
|
|
||||||
\`\`\`bash
|
|
||||||
docker logs git_sync_dev --follow
|
docker logs git_sync_dev --follow
|
||||||
\`\`\`
|
3. Manually Trigger a Sync
|
||||||
|
|
||||||
### 3. Manually Trigger a Sync
|
|
||||||
|
|
||||||
To sync immediately, use the manual push script:
|
To sync immediately, use the manual push script:
|
||||||
|
bash
|
||||||
\`\`\`bash
|
|
||||||
./scripts/manual-push-git-sync.sh
|
./scripts/manual-push-git-sync.sh
|
||||||
\`\`\`
|
4. Verify Sync
|
||||||
|
GitHub: Check https://github.com/mrhavens/mirror-repo.
|
||||||
### 4. Verify Sync
|
Forgejo: Check http://localhost:3000/mrhavens/mirror-repo.
|
||||||
|
Internet Archive: Check fold-stack-git-mirror on archive.org.
|
||||||
- **GitHub**: Check `https://github.com/mrhavens/mirror-repo`.
|
Web3.storage: Enable in remotes.conf and check fold-stack-git-mirror.
|
||||||
- **Forgejo**: Check `http://localhost:3000/mrhavens/mirror-repo`.
|
5. Generate a Sync Report
|
||||||
- **Internet Archive**: Check `fold-stack-git-mirror` on archive.org.
|
|
||||||
- **Web3.storage**: Enable in `remotes.conf` and check `fold-stack-git-mirror`.
|
|
||||||
|
|
||||||
### 5. Generate a Sync Report
|
|
||||||
|
|
||||||
View the latest sync activity for each remote:
|
View the latest sync activity for each remote:
|
||||||
|
bash
|
||||||
\`\`\`bash
|
|
||||||
./scripts/report-git-sync.sh
|
./scripts/report-git-sync.sh
|
||||||
\`\`\`
|
Example Output:
|
||||||
|
```
|
||||||
**Example Output**:
|
|
||||||
\`\`\`
|
|
||||||
=================================
|
|
||||||
📊 GIT-SYNC SYNC REPORT
|
📊 GIT-SYNC SYNC REPORT
|
||||||
=================================
|
|
||||||
📅 Date: Mon May 26 22:41:00 CDT 2025
|
📅 Date: Mon May 26 22:41:00 CDT 2025
|
||||||
|
|
||||||
---------------------------------
|
|
||||||
📌 Local Repository Latest Commit
|
📌 Local Repository Latest Commit
|
||||||
---------------------------------
|
|
||||||
Commit: a1b2c3d4e5f6g7h8i9j0k1l2m3n4o5p6
|
Commit: a1b2c3d4e5f6g7h8i9j0k1l2m3n4o5p6
|
||||||
Message: Added new feature
|
Message: Added new feature
|
||||||
Time: Mon May 26 22:40:00 CDT 2025
|
Time: Mon May 26 22:40:00 CDT 2025
|
||||||
|
|
||||||
---------------------------------
|
|
||||||
📌 Latest Sync Activity by Remote
|
📌 Latest Sync Activity by Remote
|
||||||
---------------------------------
|
|
||||||
Remote: github (git, git@github.com:mrhavens/mirror-repo.git)
|
Remote: github (git, git@github.com:mrhavens/mirror-repo.git)
|
||||||
Last Synced Commit: a1b2c3d4e5f6g7h8i9j0k1l2m3n4o5p6
|
Last Synced Commit: a1b2c3d4e5f6g7h8i9j0k1l2m3n4o5p6
|
||||||
Commit Message: Added new feature
|
Commit Message: Added new feature
|
||||||
Timestamp: [Mon May 26 22:41:00 CDT 2025]
|
Timestamp: [Mon May 26 22:41:00 CDT 2025]
|
||||||
✅ Status: Successfully synced
|
✅ Status: Successfully synced
|
||||||
|
|
||||||
Remote: forgejo (git, git@localhost:2222/mrhavens/mirror-repo.git)
|
Remote: forgejo (git, git@localhost:2222/mrhavens/mirror-repo.git)
|
||||||
Last Synced Commit: a1b2c3d4e5f6g7h8i9j0k1l2m3n4o5p6
|
Last Synced Commit: a1b2c3d4e5f6g7h8i9j0k1l2m3n4o5p6
|
||||||
Commit Message: Added new feature
|
Commit Message: Added new feature
|
||||||
Timestamp: [Mon May 26 22:41:01 CDT 2025]
|
Timestamp: [Mon May 26 22:41:01 CDT 2025]
|
||||||
✅ Status: Successfully synced
|
✅ Status: Successfully synced
|
||||||
|
|
||||||
Remote: ia (rclone, ia:fold-stack-git-mirror)
|
Remote: ia (rclone, ia:fold-stack-git-mirror)
|
||||||
Last Synced Bundle: repo-1716777660.bundle
|
Last Synced Bundle: repo-1716777660.bundle
|
||||||
Timestamp: [Mon May 26 22:41:02 CDT 2025]
|
Timestamp: [Mon May 26 22:41:02 CDT 2025]
|
||||||
✅ Status: Successfully synced
|
✅ Status: Successfully synced
|
||||||
\`\`\`
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -322,104 +219,59 @@ Timestamp: [Mon May 26 22:41:02 CDT 2025]
|
|||||||
|
|
||||||
If sync fails, run the diagnostic script:
|
If sync fails, run the diagnostic script:
|
||||||
|
|
||||||
\`\`\`bash
|
```bash
|
||||||
./scripts/diagnose-git-sync.sh
|
./scripts/diagnose-git-sync.sh
|
||||||
\`\`\`
|
Example Output:
|
||||||
|
```
|
||||||
**Example Output**:
|
|
||||||
\`\`\`
|
|
||||||
=================================
|
|
||||||
🩺 GIT-SYNC COMPREHENSIVE DIAGNOSTICS
|
🩺 GIT-SYNC COMPREHENSIVE DIAGNOSTICS
|
||||||
=================================
|
|
||||||
📅 Date: Mon May 26 22:41:00 CDT 2025
|
📅 Date: Mon May 26 22:41:00 CDT 2025
|
||||||
|
|
||||||
---------------------------------
|
|
||||||
📌 SSH Keys Check
|
📌 SSH Keys Check
|
||||||
---------------------------------
|
|
||||||
✅ /config/git-sync/secrets/github.key exists.
|
✅ /config/git-sync/secrets/github.key exists.
|
||||||
✅ /config/git-sync/secrets/github.key has correct permissions (600).
|
✅ /config/git-sync/secrets/github.key has correct permissions (600).
|
||||||
✅ /config/git-sync/secrets/forgejo.key exists.
|
✅ /config/git-sync/secrets/forgejo.key exists.
|
||||||
✅ /config/git-sync/secrets/forgejo.key has correct permissions (600).
|
✅ /config/git-sync/secrets/forgejo.key has correct permissions (600).
|
||||||
|
|
||||||
---------------------------------
|
|
||||||
📌 Remote Connectivity Test
|
📌 Remote Connectivity Test
|
||||||
---------------------------------
|
|
||||||
Testing github (git)...
|
Testing github (git)...
|
||||||
✅ github connectivity test passed.
|
✅ github connectivity test passed.
|
||||||
Testing forgejo (git)...
|
Testing forgejo (git)...
|
||||||
✅ forgejo connectivity test passed.
|
✅ forgejo connectivity test passed.
|
||||||
Testing ia (rclone)...
|
Testing ia (rclone)...
|
||||||
❌ ia connectivity test failed. Check rclone.conf or credentials.
|
❌ ia connectivity test failed. Check rclone.conf or credentials.
|
||||||
\`\`\`
|
|
||||||
|
|
||||||
### 2. Common Issues and Fixes
|
### 2. Common Issues and Fixes
|
||||||
|
|
||||||
- **Container Not Running**:
|
- **Container Not Running**:
|
||||||
\`\`\`bash
|
```bash
|
||||||
./scripts/down-dev.sh && ./scripts/up-dev.sh
|
./scripts/down-dev.sh && ./scripts/up-dev.sh
|
||||||
\`\`\`
|
SSH Key Issues:
|
||||||
- **SSH Key Issues**:
|
Verify permissions: chmod 600 config/git-sync/secrets/*.
|
||||||
- Verify permissions: `chmod 600 config/git-sync/secrets/*`.
|
Test connectivity: ssh -i ~/.ssh/github_key -T git@github.com.
|
||||||
- Test connectivity: `ssh -i ~/.ssh/github_key -T git@github.com`.
|
Rclone Remote Fails:
|
||||||
- **Rclone Remote Fails**:
|
Reconfigure Rclone: rclone config.
|
||||||
- Reconfigure Rclone: `rclone config`.
|
Verify remotes: rclone listremotes --config config/git-sync/rclone.conf.
|
||||||
- Verify remotes: `rclone listremotes --config config/git-sync/rclone.conf`.
|
Logs Missing:
|
||||||
- **Logs Missing**:
|
Check volume permissions: chmod -R 775 volumes/logs && chown -R 1000:1000 volumes/logs.
|
||||||
- Check volume permissions: `chmod -R 775 volumes/logs && chown -R 1000:1000 volumes/logs`.
|
📚 Advanced Configuration
|
||||||
|
1. Enable Web3.storage Sync
|
||||||
---
|
Edit config/git-sync/remotes.conf to enable Web3.storage:
|
||||||
|
|
||||||
## 📚 Advanced Configuration
|
|
||||||
|
|
||||||
### 1. Enable Web3.storage Sync
|
|
||||||
|
|
||||||
Edit `config/git-sync/remotes.conf` to enable Web3.storage:
|
|
||||||
|
|
||||||
\`\`\`
|
|
||||||
web3|rclone|web3:fold-stack-git-mirror|1
|
web3|rclone|web3:fold-stack-git-mirror|1
|
||||||
\`\`\`
|
2. Add a New Remote (e.g., S3)
|
||||||
|
Add a new remote to remotes.conf using Rclone:
|
||||||
### 2. Add a New Remote (e.g., S3)
|
|
||||||
|
|
||||||
Add a new remote to `remotes.conf` using Rclone:
|
|
||||||
|
|
||||||
\`\`\`
|
|
||||||
s3|rclone|s3:fold-stack-git-mirror|1
|
s3|rclone|s3:fold-stack-git-mirror|1
|
||||||
\`\`\`
|
Configure the s3 remote in config/git-sync/rclone.conf using rclone config.
|
||||||
|
3. Adjust Sync Interval
|
||||||
Configure the `s3` remote in `config/git-sync/rclone.conf` using `rclone config`.
|
Edit config/git-sync/.env to change the sync interval:
|
||||||
|
|
||||||
### 3. Adjust Sync Interval
|
|
||||||
|
|
||||||
Edit `config/git-sync/.env` to change the sync interval:
|
|
||||||
|
|
||||||
\`\`\`
|
|
||||||
SYNC_INTERVAL=60 # Check every 60 seconds
|
SYNC_INTERVAL=60 # Check every 60 seconds
|
||||||
\`\`\`
|
|
||||||
|
|
||||||
Restart the service:
|
Restart the service:
|
||||||
|
bash
|
||||||
\`\`\`bash
|
|
||||||
docker compose -f docker-compose.dev.yml stop git-sync
|
docker compose -f docker-compose.dev.yml stop git-sync
|
||||||
docker compose -f docker-compose.dev.yml up -d git-sync
|
docker compose -f docker-compose.dev.yml up -d git-sync
|
||||||
\`\`\`
|
4. Enable Commit Signing (Future Feature)
|
||||||
|
|
||||||
### 4. Enable Commit Signing (Future Feature)
|
|
||||||
|
|
||||||
To enable GPG commit signing (not yet implemented), set:
|
To enable GPG commit signing (not yet implemented), set:
|
||||||
|
|
||||||
\`\`\`
|
|
||||||
SIGN_COMMITS=true
|
SIGN_COMMITS=true
|
||||||
\`\`\`
|
|
||||||
|
|
||||||
You’ll need to:
|
You’ll need to:
|
||||||
- Add GPG keys to the container.
|
Add GPG keys to the container.
|
||||||
- Update `entrypoint.sh` to sign commits using `git`.
|
Update entrypoint.sh to sign commits using git.
|
||||||
|
📅 Last Updated
|
||||||
|
This README was last updated on May 26, 2025, at 10:54 PM CDT.
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 📅 Last Updated
|
|
||||||
|
|
||||||
This README was last updated on **May 26, 2025, at 10:41 PM CDT**.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|||||||
Reference in New Issue
Block a user