GIT

Configure git:

git config --global user.name "Your Name Comes Here"
git config --global user.email you@yourdomain.example.com
git config --global color.diff auto
git config --global color.status auto
git config --global color.branch auto
git config --global core.autocrlf false
git config --global core.savecrlf false 
git config --global diff.renames true
git config --global merge.renameLimit 999999
git config --global diff.renameLimit 999999
git config --global core.excludesfile ~/.gitignore_global

You can now edit the global ignore file:

~/.gitignore_global
# Ignore project files from PHPStorm
.idea/*.xml
.idea/.name
.idea/scopes
.idea/*.iml

Some nice settings:

git config --global alias.st status
git config --global alias.ci commit
git config --global alias.br branch
git config --global alias.co checkout
git config --global alias.df diff
git config --global alias.undolast 'reset --soft HEAD^'
git config --global alias.datetag '!git tag `date "+%Y_%m_%d_%H%M"`'
git config --global branch.autosetuprebase always
git config --global push.default current

Checkout the first version with git:

git clone idefix.fechner.net:masterthesis.git masterthesis

Have several copies on several places and sync it with the master server (like svn):

mkdir repo
cd repo
git init repo
git pull ssh://user@server/dir/
git remote add origin git@server:pluginname.git
git config branch.master.remote 'origin'
git config branch.master.merge 'refs/heads/master'
git push origin master:refs/heads/master

Remove remote Branch

git push origin :branchname

Split one Repo into Several

We have migrated now a repo but it contains several directories which should be splitted into seperate repos. The structure looks like:

git-root/
         AutoInvite
         phpraider

This can be done with filter-branch. At first checkout the repo as often as you want to split it:

git clone git@server:auto_invite AutoInvite
git clone git@server:auto_invite AutoInvite_phpRaider

Now we remove everything from the repo which should not be in:

cd AutoInvite
git filter-branch -f --tree-filter 'rm -Rf phpraider' --prune-empty -- --all
git gc
cd ..
cd AutoInvite_phpRaider
git filter-branch -f --tree-filter 'rm -Rf AutoInvite' --prune-empty -- --all
git gc
cd ..

Now we rewrite the root directory:

cd AutoInvite
git filter-branch -f --subdirectory-filter AutoInvite -- --all
git gc
cd ..
cd AutoInvite_phpRaider
git filter-branch -f --subdirectory-filter phpraider -- --all
git gc
cd ..

Push the new repos to a gitolite server:

cd AutoInvite
git push --all git@localhost:AutoInvite
cd ..
cd AutoInvite_phpRaider
git push --all git@localhost:AutoInvite_phpRaider
cd ..

Remove everything from your local repository:

git for-each-ref --format="delete %(refname)" refs/original | git update-ref --stdin
git reflog expire --expire=now --all
git gc --prune=now

Better Tool

Maybe a better tool is: https://github.com/newren/git-filter-repo

Only keep one folder:

git-filter-repo --force --path software/telegram\ analyser

To remove the folder in the other repository:

git-filter-repo --force --path software/telegram\ analyser --invert-paths

To move the newly filtered project into ROOT path you can execute:

git-filter-repo --path-rename software/telegram\ analyser/:

Modify First Commit

The problem here is that the first commit cannot be changed with the rebase command. So we have to create a new first commit befor we can modify the real first commit:

git symbolic-ref HEAD refs/heads/newroot
git rm --cached -r .
git clean -f -d
# touch .gitignore && git add .gitignore # if necessary
git commit --allow-empty -m 'Initial commit'
git rebase --onto newroot --root master
git branch -d newroot

Now we can change commits with rebase -i as usual.

Merge Two Git Repositories

git remote add repotomerge /path/to/repo
git fetch repotomerge 
git merge repotomerge/<branch to merge>
git gc

Merge Two Git Repositories into one with own Sub Folders

We have two git repositories called subfolder1.git and subfolder2.git, both repositories will contain several branches and we want to build up a complete new structure in the way:

base directory
 |- subfolder1 (from repository subfolder1)
 |- subfolder2 (from repository subfolder2)
 |- more folder (newly added)
- file1 (newly added)

The repositories are available via git clone from:

http://server/subfolder1.git
http://server/subfolder2.git

We use the following script to checkout the repository and rewrite the path and merge everything into one repoistory with complete history and all branches:

mergeGitRepos.sh
#!/bin/bash
# (c) 2013 Matthias Fechner


me=$(basename "$0")

TMP=$(mktemp -d /tmp/$me.XXXXXXX)
echo
echo "building new repo in $TMP"
echo

set -e

cd "$TMP"
mkdir new-repo
cd new-repo
        git init
        cd ..

x=0
while [ $# -gt 0 ]
do
        repo="$1"
        shift
        dirname=$(basename "$repo" | sed -e 's/\s/-/g')
        dirname=$(basename "$dirname" | sed -e 's/.git$//')
        echo "Clone $repo"
        git clone --bare "$repo" "$dirname"

        cd "$dirname"
                echo "Checkout all branches"
                for remote in `git branch -r | grep -v master `; do git checkout --track $remote ; done
                git checkout master
                echo "Remove reference to origin, so we cannot push by accident"
                git remote rm origin
                echo "Move all files and directories to new location"
                git filter-branch --index-filter 'git read-tree --empty; git read-tree --prefix="$dirname/" "$GIT_COMMIT"' -- --all
                cd ..

        cd new-repo
                git pull --no-commit ../$dirname
                [ $x -gt 0 ] && git commit -m "merge made by $me"
                cd ..

        x=$(( x + 1 ))
done

The script is named mergeGitRepos.sh and is executed this way:

mergeGitRepos.sh http://server/subfolder1.git http://server/subfolder2

After it is done. you can copy your missing folder (more folders) and your missing files and commit them as usal.

Second Approach

The script failed and do not what I wanted. New approach is to filter every repository to get the correct path there.

At first we copy the repository on the server so we work on a test repository, that is important, because we will overwrite it several times to get everything merged.

cp -r repo.git repo_save.git

Now the real work:

git clone http://server/repo_save.git
cd repo_save
for remote in `git branch -r | grep -v master `; do git checkout --track $remote ; done
git checkout master
git remote rm origin
git filter-branch --index-filter 'git read-tree --empty; git read-tree --prefix="subfolder/" "$GIT_COMMIT"' -- --all
git push -f --all http://server/repo_save.git
git push --tags http://server/repo_save.git
cd ..
rm -rf repo_save

Redo this for every repository. Now all repositories should have there correct folder displayed for all branches.

Now we combine them to one repository:

mkdir new-repo
cd new-repo
git init
git remote add origin http://server/repo1_save.git
git fetch --all
git fetch --all -t
git pull
git checkout master
for remote in `git branch -r | grep -v master `; do git checkout --track $remote ; done
git checkout master
git remote rm origin


git remote add origin http://server/repo2_save.git
git fetch --all
git fetch --all -t
git merge origin/master
... merge all branches ....
git remote rm origin

git remote add origin http://server/new-repo.git
git push --all
git push --tags
cd ..
rm -rf new-repo

Now you have your new repository on the server in new-repo.git.

Clone it, test it thate everything like you expected it. If everything is fine, remove all backup and you are done.

Delete files from git history

You have a project and you want to make it public but you commited password to your repository? Lets say the passwords are in the file bin/config, so we can do the following:

git filter-branch --index-filter 'git rm -r --cached --ignore-unmatch bin/config' HEAD

Bind

DNSSec

DNSSec for Caching DNS Servers

Add the following into your named.conf:

options {
     dnssec-enable yes;
     dnssec-validation auto;
};

Restart your DNS server now with:

/etc/rc.d/named restart

To test it you should execute the command and the RRSIG should be displayed:

dig +dnssec isc.org soa

You should see in the flags ad that ensures that everything is fine:

;; flags: qr rd ra ad; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 1

DNSSec for Servers

We use a complete new feature of bind 9.9.5-P1. Bind will handle all the key and signing for us. So it is not necessary to resign your zones after one month, bind will do that automatically for you.

I assume you have your configuration in folder:

/usr/local/etc/namedb

Zonefiles are in:

/usr/local/etc/namedb/master

Keyfiles in:

/usr/local/etc/namedb/keys

To start:

mkdir -p /usr/local/etc/namedb/keys
chown bind:bind /usr/local/etc/namedb/keys
chown bind:bind /usr/local/etc/namedb/master

Edit your named.conf:

options {
...
        // enable dnssec
        dnssec-enable yes;
        dnssec-validation auto;
        managed-keys-directory "/usr/local/etc/namedb/working/";
        key-directory "/usr/local/etc/namedb/keys/";
        allow-new-zones yes;
}

logging {
        channel log_zone_transfers {
                file "/var/log/named/named-axfr.log";
                print-time yes;
                print-category yes;
                print-severity yes;
                };
        channel named_log {
                file "/var/log/named/named.log" versions 3 size 2m;
                severity info;
                print-severity yes;
                print-time yes;
                print-category yes;
        };
        category xfer-in { log_zone_transfers; };
        category xfer-out { log_zone_transfers; };
        category notify { log_zone_transfers; };
        category default { named_log; };
        category lame-servers { null; };
};

// define DNSSEC KASP
dnssec-policy one-year-zsk {
        keys {
                zsk lifetime 365d algorithm ecdsa256;
                ksk lifetime unlimited algorithm ecdsa256;
        };
};

Define your zone like this:

zone "fechner.net" {
        type master;
        file "/usr/local/etc/namedb/master/fechner.net/fechner.net";
        allow-transfer { inwx; };
        dnssec-policy "one-year-zsk";
        inline-signing yes;
};

Bind will now automatically create DNS keys for you and take care of renewal.

If everything is correct you should see your dnskey with:

dig @localhost dnskey fechner.net.

To display the zone including the dynamically added key and signatures execute:

cd /usr/local/etc/namedb/master/fechner.net
named-checkzone -D -f raw -o - fechner.net fechner.net.signed | less

The KSK has ID 257 and ZSK has 256.

dig +multi fechner.net DNSKEY
...
fechner.net.            3600 IN DNSKEY 256 3 13 (
                                yZQLC3g4RnT2knGmQBJABr9PxjnhcIZuY2mpFT+mb2M2
                                VVWWP+EY//A/fbqCoqfZMneUmVCz+6rzSRCg7xPNlg==
                                ) ; ZSK; alg = ECDSAP256SHA256 ; key id = 31203
fechner.net.            3600 IN DNSKEY 257 3 13 (
                                /W0+wjfR0nKcRiyL3tYYjz1QHffK0ynn5/b2N6oYDbE8
                                zRzoU11XkeQ8pX8lok66EcRFUQtkyRySw65G8Bbsdg==
                                ) ; KSK; alg = ECDSAP256SHA256 ; key id = 15520
...

So the keyid for the KSK 15520. We use this keyid in the next command to get the DS which is required for the parent for the chain of trust.

To get the fingerprint of your signing key we can execute one of the following command this:

dig @localhost dnskey fechner.net | dnssec-dsfromkey -f - fechner.net
# or (13 is the algo, 15520 is the keyid)
dnssec-dsfromkey Kfechner.net.+013+15520.key 

Register DNSKEY at Registrar

Example for INWX

For INWX go in the webinterface to Nameserver->DNSSEC and click on DNSSEC hinzufügen. Remove checkbox for automatischer Modus.

Fill your domain: fmdata.net.

To get the keyid for the KSK you can use:

dig dnskey fmdata.net. +multi
;; ANSWER SECTION:
fmdata.net.             3411 IN DNSKEY 256 3 13 (
                                WcoWkUyFAX+51FQGPI70nyTHPWagCJZZq/GmhKg8sxK2
                                ZPQh6Cu+dpfLrAWxr8udthyJeFCscaPsv1+3mMVT2A==
                                ) ; ZSK; alg = ECDSAP256SHA256 ; key id = 38157
fmdata.net.             3411 IN DNSKEY 257 3 13 (
                                sd2MViZMwa7hpKUMCKlZWFMwUJVYO31q+Fzte9IFUHVe
                                wQwvbdb9Ah9Si9mV6lSLqJOPvews+ytYoICE/7MmbQ==
                                ) ; KSK; alg = ECDSAP256SHA256 ; key id = 7947

So the keyid we need here for the KSK is 7947. You have now two possibilities to get the record (I suggest both and make sure they match): From your keys directory

cat Kfmdata.net.+013+07947.key
...
fmdata.net. 3600 IN DNSKEY 257 3 13 sd2MViZMwa7hpKUMCKlZWFMwUJVYO31q+Fzte9IFUHVewQwvbdb9Ah9S i9mV6lSLqJOPvews+ytYoICE/7MmbQ==

Using dig (make sure you take the 257!):

dig dnskey fmdata.net. +dnssec
...
fmdata.net.             3201    IN      DNSKEY  257 3 13 sd2MViZMwa7hpKUMCKlZWFMwUJVYO31q+Fzte9IFUHVewQwvbdb9Ah9S i9mV6lSLqJOPvews+ytYoICE/7MmbQ==
...

Make sure you remove the TTL so use the following line:

fmdata.net. IN DNSKEY 257 3 13 sd2MViZMwa7hpKUMCKlZWFMwUJVYO31q+Fzte9IFUHVewQwvbdb9Ah9S i9mV6lSLqJOPvews+ytYoICE/7MmbQ==

Put this line into the first field (DNSKEY RR:).

To get the DS:

dnssec-dsfromkey Kfmdata.net.+013+07947.key
fmdata.net. IN DS 7947 13 2 05F14B98499079F564FA8DFAAAC06051F9929B8AB3921F2FA354E17C39F9CBA6

Compare this with:

dig dnskey fmdata.net. +dnssec | dnssec-dsfromkey -f - fmdata.net.
fmdata.net. IN DS 7947 13 2 05F14B98499079F564FA8DFAAAC06051F9929B8AB3921F2FA354E17C39F9CBA6

If the match, insert this line into the second field in the webinterface (DS Record:).

Check

To read the content of the fechner.net.signed:

named-checkzone -D -f raw -o - fechner.net fechner.net.signed

DANE

Postfix

cd /usr/local/etc/apache24/ssl_keys
openssl x509 -in newcert.pem -outform DER |openssl sha256

Take the fingerprint and create a new line in your zone file:

_25._tcp.<domain>. 1H IN TLSA 3 0 1 <fingerprint>
_465._tcp.<domain>. 1H IN TLSA 3 0 1 <fingerprint>

or with sha512:

cd /usr/local/etc/apache24/ssl_keys
openssl x509 -in newcert.pem -outform DER |openssl sha512
_25._tcp.<domain>. 1H IN TLSA 3 0 2 <fingerprint>
_465._tcp.<domain>. 1H IN TLSA 3 0 2 <fingerprint>

SSH

cd /usr/ports/dns/sshfp
make install clean
sshfp idefix.fechner.net

Take the line and add it to your zonefile:

idefix.fechner.net IN SSHFP 1 1 26282825A61D967F751BB74E8B7930FCF3A25120
idefix.fechner.net IN SSHFP 2 1 963DDFF48B3FCCC379AC07D5A7759C89EA2B45B7

Make sure to add a dot after the hostname.

Check records

https://de.ssl-tools.net

DNSSEC for clients starting FreeBSD 10

echo 'local_unbound_enable="YES"' >> /etc/rc.conf

Check every nameserver from /etc/resolv.conf:

drill -S fechner.net @213.133.98.98

Start unbound to generate new config files:

service local_unbound onestart

Recheck resolving:

drill -S fechner.net
;; Chasing: fechner.net. A
Warning: No trusted keys specified


DNSSEC Trust tree:
fechner.net. (A)
|---fechner.net. (DNSKEY keytag: 37748 alg: 10 flags: 256)
    |---fechner.net. (DNSKEY keytag: 64539 alg: 10 flags: 257)
    |---fechner.net. (DS keytag: 64539 digest type: 1)
    |   |---net. (DNSKEY keytag: 6647 alg: 8 flags: 256)
    |       |---net. (DNSKEY keytag: 35886 alg: 8 flags: 257)
    |       |---net. (DS keytag: 35886 digest type: 2)
    |           |---. (DNSKEY keytag: 22603 alg: 8 flags: 256)
    |               |---. (DNSKEY keytag: 19036 alg: 8 flags: 257)
    |---fechner.net. (DS keytag: 64539 digest type: 2)
        |---net. (DNSKEY keytag: 6647 alg: 8 flags: 256)
            |---net. (DNSKEY keytag: 35886 alg: 8 flags: 257)
            |---net. (DS keytag: 35886 digest type: 2)
                |---. (DNSKEY keytag: 22603 alg: 8 flags: 256)
                    |---. (DNSKEY keytag: 19036 alg: 8 flags: 257)
You have not provided any trusted keys.
;; Chase successful

Manage your Zones with git and nsdiff / nsupdate (WIP)

The idea here is that you have all your zone data on another server in a directory that is managed via git. Changes can be applied directly via scripts to a server or can be pushed to gitlab and are automatically deployed via a pipeline.

It is only necessary to create a basic zonefile on the server and create a key that allows the remote update of the zone.

The DNSSEC keys, signing the zones, taking care of keys is all transparently done be the server.

So you can focus on the real work and get rid of all the administrative overhead.

Also using DNS based verification for wildcard certificates is possible

Configure the server

Create a key that is used to authenticate against the DNS server.

We use for the key name the FQDN of client and server and separate them with a -. Execute on the DNS Server:

cd /usr/local/etc/namedb
tsig-keygen clientFQDN-serverFQDN. >> keys.conf
chown bind:bind keys.conf
chmod 640 keys.conf

Now we edit named.conf and include the key just generated. I manage my master zone in an extra file, we include here too:

/usr/local/etc/namedb/named.conf
...
include "/usr/local/etc/namedb/keys.conf";
include "/usr/local/etc/namedb/named.zones.master";
...

Define the zone:

/usr/local/etc/namedb/named.zones.master
zone "fechner.net" {
        type master;
        file "/usr/local/etc/namedb/master/fechner.net/fechner.net";
        dnssec-policy "one-year-zsk";
        inline-signing yes;
        allow-transfer { key clientFQDN-serverFQDN.; };
        allow-update { key clientFQDN-serverFQDN.; };
};

Create the zone file and add a very basic definition:

mkdir -p /usr/local/etc/namedb/master/fechner.net

Edit the zone file:

/usr/local/etc/namedb/master/fechner.net
$TTL 1d ; 1 day
@                       IN SOA  ns.fechner.net. hostmaster.fechner.net. (
                                2023070201 ; serial
                                12h        ; refresh (12 hours)
                                2h         ; retry (2 hours)
                                3w         ; expire (3 weeks)
                                1d         ; minimum (1 day)
                                )

                        NS      ns.fechner.net.
                        NS      ns.catacombs.de.
ns                      A       89.58.45.13
ns                      AAAA    2a03:4000:67:cc1::2

Restart bind with:

service named restart

Configure the Client

The Client can be on the server and/or on another host. You should just ensure that you keep this directory or repository in sync, for this we use git.

I will not explain git here, I expect you know, if not, there are nice manuals existing. With normal user on your local computer the zonefiles are stored, I name the folder now zonefiles-fqdn-nameserver

At first, we need to install the tools required:

pkg install p5-DNS-nsdiff git
cd git/gitlab.fechner.net/zonefiles-fqdn-nameserver
mkdir fechner.net
touch fechner.net/fechner.net

Now edit you zone file that it matches your requirements.

You can diff your zone now to the zone on the server with:

#usage: nsdiff [options] <zone> [old] [new]
nsdiff  -k ../.key -S date -d fechner.net fechner.net

You can verify now if the changes are making sense.

If it makes sense you can apply it with:

nsdiff  -k ../.key -S date -d fechner.net fechner.net |nsupdate -k ../.key -d

OLD DNSSec for Servers

You do NOT want to do it this way anymore.

http://alan.clegg.com/files/DNSSEC_in_6_minutes.pdf

Create the ZSK:

dnssec-keygen -a RSASHA1 -b 1024 -n ZONE idefix.lan

Create the KSK:

dnssec-keygen -a RSASHA1 -b 4096 -n ZONE -f KSK idefix.lan

Add the keys to your zone file:

cat K*.key >> idefix.lan

Sign the zone:

dnssec-signzone -N INCREMENT -l dlv.isc.org. idefix.lan

Now change the file loaded to the signed one:

zone "idefix.lan" IN {
    file "/etc/namedb/master/idefix.lan.signed";
};

Reload the zone with:

rndc reconfig
rndc flush

Automation on the server

We start to install a toolset to automate all the resigning and recreation (rolling) of the keys.

cd /usr/ports/security/softhsm
make install
make clean
cd /usr/ports/dns/opendnssec
make install
make clean

Configure some basic settings like pin in /usr/local/etc/opendnssec/conf.xml. Also set in section Signer:

<NotifyCommand>/usr/sbin/rndc reload %zone</NotifyCommand>

Now we create the key holding database:

softhsm --init-token --slot 0 --label "OpenDNSSEC"

Enter the pin used in the config.xml.

Setup the database with:

ods-ksmutil setup

Create a start-up file that start opendnssec everytime you start your server. Create for this the file /usr/local/etc/rc.d/opendnssec:

  - !/bin/sh

  -  PROVIDE: opendnssec
  -  REQUIRE: named

  - 
  -  Add the following line to /etc/rc.conf to enable radvd:
  - 
  -  opendnssec_enable="YES"
  - 

. /etc/rc.subr

name=opendnssec
rcvar=`set_rcvar`

pidfile=/usr/local/var/run/opendnssec/signerd.pid
command="/usr/local/sbin/ods-control"
command_args="start"

load_rc_config $name
> ${opendnssec_enable="no"}

run_rc_command "$1"

And make it executeable with:

chmod +x /usr/local/etc/rc.d/opendnssec

Now enable the startup script in /etc/rc.conf with:

opendnssec="YES"

and start it with

/usr/local/etc/rc.d/opendsnsec start

Check the logfile /var/log/messages that everything is fine.

Now add the zones with:

ods-ksmutil zone add --zone example.com

https://sys4.de/de/blog/2014/05/24/einen-tlsa-record-fuer-dane-mit-bind-9-publizieren/

Altium

use the following ignore definition:

__Previews/
History/
* Logs/*
Project Output *
*.DSNlck

Changelog

To create a changelog:

git log change_20151110..change_20160114  --pretty=format:'- %s' --reverse |grep -v Merge

To create a html output:

git log change_20151110..change_20160114  --pretty=format:'<li> <a href="http://server/?p=repository;a=commit;h=%H">view commit &bull;</a> %s</li> ' --reverse |grep -v Merge

Git on Windows

To install git for windows go to website: https://git-for-windows.github.io/ and download the latest version. I tested it with Git-2.3.5.8-dev-preview-32-bit.exe.

Apache virtual host configuration:

gitweb.conf
<VirtualHost *:80>
SetEnv GIT_PROJECT_ROOT "D:/Gitrepos"
SetEnv GIT_HTTP_EXPORT_ALL
ScriptAliasMatch \
	"(?x)^/(.*/(HEAD | \
			info/refs | \
			objects/(info/[^/]+ | \
				[0-9a-f]{2}/[0-9a-f]{38} | \
				pack/pack-[0-9a-f]{40}\.(pack|idx)) | \
			git-(upload|receive)-pack))$" \
			"D:/Git/mingw64/libexec/git-core/git-http-backend.exe/$1"

DocumentRoot "D:/Git/mingw64/share/gitweb"
<Directory "D:/Git/mingw64/share/gitweb">
	DirectoryIndex gitweb.cgi
	<Files *.cgi>
		SetHandler cgi-script
	</Files>
	Options ExecCGI FollowSymLinks
	SetEnv  GITWEB_CONFIG "d:/Gitrepos/gitweb.conf"
	AuthType Digest
	AuthName "Git Password Required"
	AuthDigestDomain /git/
	AuthUserFile "D:/Gitrepos/htdigest_passwd"
	Require valid-user
</Directory>
<Directory />
	AuthType Digest
	AuthName "Git Password Required"
	AuthDigestDomain /git/
	AuthUserFile "D:/Gitrepos/htdigest_passwd"
	Require valid-user
</Directory>
</VirtualHost>

Edit the file d:\Git\mingw64\share\gitweb\gitweb.cgi and change the first line to:

#!D:/Strawberry/perl/bin/perl.exe

And modify the function get_file_owner:

sub get_file_owner {
	my $path = shift;

	my $owner = "Matthias Fechner";
	return to_utf8($owner);
}

Create a config file for gitweb:

d:\gitrepos\gitweb.conf
our $GIT = "d:/Git/bin/git";
our $projectroot= "d:/Gitrepos";
our $git_temp = "/d";
our $projects_list = "d:/Gitrepos/projects.list";
our $export_ok = "git-daemon-export-ok";
our $strict_export = "false";

If you get an error message that CGI.pm is missing you can install it manually using the cpan shell. At first we define a possibly required proxy, so open a git shell and do:

export http_proxy=http://proxyhost:proxyport/
export ftp_proxy=http://proxyhost:proxyport/
cpan install CGI

If you have problems with proxy authentication you can configure user and password also manually with:

cpan
o conf http_proxy proxyhost
o conf ftp_proxy proxyhost
o conf proxy_user username
o conf proxy_passw password
o conf commit
exit

Automatic Deploy

We have an application running on tomcat and the application is managed by a git repository. As application server we use tomcat on a windows server having git installed. Now I would like to have a simple script I fire that stops the appliation server, pull changes, checkout it, remove all files not included in commits, rollout configuration files and restart tomcat.

Configuration files are stored with file extension .STG, .STG1, .PRD, .PRD1, .PRD2, .PRD3… and the normal configuration file is not part of git repository. So we can have different configuration files in the git repository and the script will take the correct one and copy it accordingly.

We do this using a simple shell script. For this create a link that has as target the following link defined:

C:\WINDOWS\system32\cmd.exe /c ""C:\Program Files\Git\usr\bin\sh.exe" --login -i updateapp.sh"

Now go into your home directory and create a file called updateapp.sh:

updateapp.sh
system="STG"
node="STG2"

net stop Tomcat7
echo Clear cache files
rm -rf /d/Archibus/apps/webcentral/tools/tomcat/webapps/archibus/schemaCompiled
rm -rf /d/Archibus/apps/webcentral/tools/tomcat/work/Catalina

echo
echo Update archibus folder
cd /d/Archibus/apps/webcentral/tools/tomcat/webapps/archibus
git checkout -f master
git fetch
git reset --hard origin/master
git clean -fd

echo
echo Update configuration files
for i in $(find . -type f -name "*.$system");
do
  cp -v "$i" "${i%.$system}"
done

for i in $(find . -type f -name "*.$node");
do
  cp -v "$i" "${i%.$node}"
done

echo
echo Update archibus_help folder
cd /d/Archibus/apps/webcentral/tools/tomcat/webapps/archibus_help
git checkout -f master
git fetch
git reset --hard origin/master
git clean -fd

echo Press a key to start Tomcat
read
net start Tomcat7
echo
echo Update finished
echo Press any key to exit
read

GitoLite

Install gitolite

Gittolite is a nice tool to manage access and git repositories with an additional admin-repository.

Server Site

Install the port by:

cd /usr/ports/devel/gitolite/
make install
make clean

Now we create the home dir of the git user:

mkdir /usr/local/git
chown git:git /usr/local/git

Set a password for the user git:

passwd git

Client Site

Now we copy our public key to the server with:

scp ~/.ssh/id_rsa.pub git@server:name.key

Now login with the user git and ssh:

ssh -o PubkeyAuthentication=no git@server

Now we setup the gitolite repo:

gitolite setup name.key
exit vi without modification on the file.
exit

Checkout the admin repository with:

git clone git@server:gitolite-admin

Do your modification, commit it and push it back to the server.

Move existing repo to gitolite

At first we create a new repo using the gitolite mechanism by editing the file gitolite.conf from the cloned gitolite-admin repository:

repo newrepo
  RW+ = user

Commit this change and push it to the server.

Now we can push our existing repo into gitolite. Go to the repo you want to push into your gitolite managed repos and insert:

git push --all git@server:newrepo

Upgrade Gitolite v3

Upgrade gitolite using normal FreeBSD upgrade (portmaster or portupgrade). After this is finished we have to upgrade the hook scripts.

Login into the server with the git account:

ssh -o PubkeyAuthentication=no git@server

Upgrade with:

gitolite setup

That’s it.

EMail Notifier

gem install git-commit-notifier

Convert SVN to GIT

First step is to create a translation file for the users. Follow here [[git:git_and_svn?s[]=checkout#checkout_a_svn_repo]] to create this author file.

Download the script [https://raw.githubusercontent.com/schwern/svn2git/master/svn2git].

Then we will clone the repository:

export repo=arinc653

git svn clone file:///usr/local/svn/${repo} -A authors.txt -s ${repo}
cd ${repo}
git branch -a -v

# you only have to execute this if you have branches or tags in the repo
svn2git --no-clone

git remote add origin git@localhost:${repo}
git gc --aggressive
git fsck --unreachable
git push --all
cd ..

rm -Rf ${repo}
rm -Rf /usr/local/svn/${repo}

Convert CVS to Git

Install cvs2svn with git support:

cd /usr/ports/devel/cvs2svn
make install
make clean
mkdir migrate-old-repos-to-git
cd !$
cp /usr/local/share/examples/cvs2svn/cvs2git-example.options cvs2git.options

Edit the file like this:

 diff -ud cvs2git.options.old cvs2git.options                                                  idefix@server
--- cvs2git.options.old 2014-06-11 13:00:54.939373946 +0200
+++ cvs2git.options     2014-06-11 13:04:39.698356333 +0200
@@ -188,7 +188,7 @@ ctx.trunk_only = False
 ctx.cvs_author_decoder = CVSTextDecoder(
     [
         #'utf8',
-        #'latin1',
+        'latin1',
         'ascii',
         ],
     #fallback_encoding='ascii'
@@ -196,7 +196,7 @@ ctx.cvs_author_decoder = CVSTextDecoder(
 ctx.cvs_log_decoder = CVSTextDecoder(
     [
         #'utf8',
-        #'latin1',
+        'latin1',
         'ascii',
         ],
     #fallback_encoding='ascii',
@@ -207,7 +207,7 @@ ctx.cvs_log_decoder = CVSTextDecoder(
 ctx.cvs_filename_decoder = CVSTextDecoder(
     [
         #'utf8',
-        #'latin1',
+        'latin1',
         'ascii',
         ],
     #fallback_encoding='ascii'
@@ -513,7 +513,7 @@ ctx.retain_conflicting_attic_files = Fal
 # (name, email).  Please substitute your own project's usernames here
 # to use with the author_transforms option of GitOutputOption below.
 author_transforms={
-    'jrandom' : ('J. Random', 'jrandom@example.com'),
-    'mhagger' : 'Michael Haggerty <mhagger@alum.mit.edu>',
-    'brane' : (u'Branko Čibej', 'brane@xbc.nu'),
-    'ringstrom' : 'Tobias Ringström <tobias@ringstrom.mine.nu>',
-    'dionisos' : (u'Erik Hülsmann', 'e.huelsmann@gmx.net'),
+    'idefix' : ('Matthias Fechner', 'spam@fechner.net'),
@@ -561,7 +561,7 @@ run_options.set_project(
     # The filesystem path to the part of the CVS repository (*not* a
     # CVS working copy) that should be converted.  This may be a
     # subdirectory (i.e., a module) within a larger CVS repository.
-    r'test-data/main-cvsrepos',
+    r'/usr/local/cvs/bericht_pra2',

     # A list of symbol transformations that can be used to rename
     # symbols in this project.
export repo=bericht_pra2

mkdir cvs2git-tmp
cvs2git --options=cvs2git.options
mkdir ${repo}.git
cd ${repo}.git
git init --bare
git fast-import --export-marks=../cvs2git-tmp/git-marks.dat < ../cvs2git-tmp/git-blob.dat
git fast-import --import-marks=../cvs2git-tmp/git-marks.dat < ../cvs2git-tmp/git-dump.dat
git gc
git remote add origin git@localhost:${repo}
git push origin
cd ..

rm -Rf ${repo}.git cvs2git-tmp
rm -Rf /usr/local/cvs/${repo}

GIT and SVN

Checkout a SVN Repo

At first we have to create a translation file for the authors:

svnuser1 = First User <user@hellospambot.com>
svnuser2 = Another User <anotheruser@whatever.com>

To get a list of users you can execute the following small shell script:

#!/bin/sh
#
# Extract information from /etc/passwd and build up a translation file for git.
# It extracts the infos to a file called authors-PID.txt
#
# (c) 2010 Matthias Fechner
#
TMPFILE=/tmp/tmp-authors-$$.tmp
EXPORTFILE=authors-$$.txt
svn -q log | grep '^r' | cut -d ' ' -f 3 | sort | uniq > $TMPFILE
rm $EXPORTFILE
for i in `cat $TMPFILE`
do
echo "Search user $i"
echo -n "$i = " >> $EXPORTFILE
USERSTRING=`grep '^'$i':' /etc/passwd | cut -d ':' -f 5`
echo "  found $USERSTRING"
FULLNAME=`echo $USERSTRING | cut -d ',' -f 1`
echo "  found $FULLNAME"
echo -n $FULLNAME >> $EXPORTFILE
EMAIL=`echo $USERSTRING | cut -d ',' -f 5`
echo " <$EMAIL>" >> $EXPORTFILE
done
rm $TMPFILE

To get only a list of all people committed into the repository:

svn -q log | grep ^r | cut -d '|' -f 2 | sort | uniq

Then we can do a clone:

git svn clone <svn repo url> -A authors.txt -s <destination dir name>

Working on it

With git-svn, you get by default a local branch named master. You should not do any work on it, only keep it up-to-date with the svn trunk branch.

git checkout master
git svn fetch
git svn rebase

If you want to do some modifications create a local branch:

git branch local-devel
git checkout local-devel

Now change the code, test it and do local commits:

fix the bug, compile, test,
git commit -a
fix the bug, compile, test,
git commit -a

If you ready and want to commit it to the remote repository we have to update our local trunk and rebase (do not use git merge, it will through away your commit messages) it:

git checkout master
git svn fetch
git svn rebase
git rebase --interactive --preserve-merges local-devel
git svn dcommit

Now we can remove our local branches:

git branch -D local-devel

Convert subversion to git with not all branches and tags

Init the new repository:

mkdir newdir
cd newdir
git svn init svn://server/path -s
git config svn.authorsfile ~/authors

Now edit .git/config

[svn-remote "svn"]
    url = svn://server
    fetch = server/trunk:refs/remotes/trunk
    branches = server/branches/{branch1, branch2}:refs/remotes/branches/*

Then clone it with:

git svn fetch

Ignore all files which are ignored by subversion:

git svn show-ignore > .gitignore

Convert all remote branches and tags to local ones by using the script svn2git :

svn2git --no-clone

Cleanup:

git gc --aggressive
git fsck --unreachable

Using GIT as Central Repository on a Server

To use a git repository on a central server to sync it between several computers we will convert it to a bare repository and place the bare repository on a place we like to share it. For this the first step is to create this bare repository with:

umask 007
cd /usr/local/gitroot
git clone --bare /tmp/repo-to-clone.git my-shared-repo.git

Finally we have to set some parameters to share it:

cd my-share-repo.git
git config core.sharedRepository 1
git config receive.denyNonFastForwards true
find objects -type d -exec chmod 02770 {} \;

Check ‘git help config’ to see what the parameters will do. If you create a new repository you can use also git init --shared my-share-repo.git.

Now can can clone the repository change files commit it and push it to the server.

Git