Home » Archive for category 'Software'

Software Archive

Fie diddle dee dee, init, and init.d

Posted April 8, 2013 By weetabix

 

On being lazy:

 

I love being lazy. In fact, it’s fair to say I take it quite seriously.

One of my favorite teachers taught me something deeply memorable one day. He taught me that being lazy is the best way to approach any job. Not just for you, but for the job itself and everyone else it affects.

Now, on the face of it this might sound like some poor advice.

Follow me along here:

 

1. The goal of being lazy is to not do more work than you have to.

2. Doing something once is less work than doing it twice.

3. To do something only once, you have to do it right the first time.

So, therefore, if you want to be lazy, you have to do the job right the first time.

– From the unforgettable Chris Venne.

 

One of the best ways to be lazy is with init scripts. Since I’m a Xubuntu/Debian fanboy, I use both upstart init and (x)init.d style scripts. Now, these can be easy, and sometimes they can be hard, especially when you don’t have useful examples. (and especially more so when half the sites reap off each other and have the same ding dong mistake. More on that later.)

To avoid confusion, when I say upstart script, I mean the newer style scripts run from /etc/init/ by Upstart. When I say init.d or init script, I mean a script run by init from /etc/init.d/. Hopefully this helps avoid confusion. (Though in the future after init is very deprecated, I will likely just call then init scripts.)

Some people reading this likely know what upstart and init/init.d scripts are. If you do not, the short version is, they are scripts run by services (called Upstart and init) that manage other services. This is how you get stuff to start and stop on boot and shutdown.

I keep around some template files, so I never have to worry too much about remembering how to do it.

 

For init we have: (Remember, you can click on the top blue line to download the file.)

/weetabytes/blob/master/initscript
#!/bin/sh
#
 
### BEGIN INIT INFO
# Provides:          <nameofservice>
# Required-Start:    $local_fs $network $remote_fs
# Required-Stop:     $local_fs $network $remote_fs
# Should-Start:      NetworkManager
# Should-Stop:       NetworkManager
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: Durr
# Description:       Herp a Durr
#
#
### END INIT INFO
 
DAEMON=<Binary to run>
#This is optional. It is used to load variables for this file.
SETTINGS=/etc/default/<same as initfilename>
 
([ -x $DAEMON ] && [ -r $SETTINGS ]) || exit 0
 
DESC="<What are you starting or stopping>"
DEFOPTS="<options to append to binary call>"
#this script is intended to start python binaries.
PYTHONEXEC="<path to python exec>"
PIDFILE=/var/run/<somedir>
#Set this to true is you decide to require a defaults file.
SETTINGS_LOADED=FALSE
 
# these are only accepted from the settings file
unset USER CONFIG HOST PORT EXTRAOPTS
 
. /lib/lsb/init-functions
 
check_retval() {
	if [ $? -eq 0 ]; then
		log_end_msg 0
		return 0
	else
		log_end_msg 1
		exit 1
	fi
}
 
is_running() {
	# returns 0 when running, 1 otherwise
    #remove pythonexec if you are calling the binary directly.
    PID="$(pgrep -f -x -u ${USER%:*} "$PYTHONEXEC $DAEMON $DEFOPTS.*")"
	RET=$?
	[ $RET -gt 1 ] && exit 1 || return $RET
}
 
load_settings() {
	if [ $SETTINGS_LOADED != "TRUE" ]; then
		. $SETTINGS
 
		[ -z "$USER" ] && {
			log_warning_msg "$DESC: not configured, aborting. See $SETTINGS";
			return 1; }
		[ -z "${USER%:*}" ] && exit 1
 
		OPTIONS="$DEFOPTS"
		[ -n "$CONFIG" ] && OPTIONS="$OPTIONS --config-file $CONFIG"
		[ -n "$HOST" ] && SERVER="$HOST" || SERVER=
		[ -n "$PORT" ] && SERVER="$SERVER:$PORT"
		[ -n "$SERVER" ] && OPTIONS="$OPTIONS --server $SERVER"
		[ -n "$EXTRAOPTS" ] && OPTIONS="$OPTIONS $EXTRAOPTS"
		SETTINGS_LOADED=TRUE
	fi
	return 0
}
 
start_bin() {
	load_settings || exit 0
	if ! is_running; then
		log_daemon_msg "Starting $DESC"
		start-stop-daemon --quiet --chuid $USER --start --exec $DAEMON -- $OPTIONS
		check_retval
		# create a pidfile; we don't use it but some monitoring app likes to have one
		[ -w $(dirname $PIDFILE) ] && \
        #remove pythonexec if you are calling the binary directly.
			pgrep -f -x -n -u ${USER%:*} "$PYTHONEXEC $DAEMON $OPTIONS" > $PIDFILE
	else
		log_success_msg "$DESC: already running (pid $PID)"
	fi
}
 
stop_bin() {
	load_settings || exit 0
	if is_running; then
		TMPFILE="$(mktemp --tmpdir <binary name>.XXXXXXXXXX)"
		[ $? -eq 0 ] || exit 1
		trap '[ -f "$TMPFILE" ] && rm -f "$TMPFILE"' EXIT
		echo "$PID" > "$TMPFILE"
		log_daemon_msg "Stopping $DESC"
		start-stop-daemon --stop --user ${USER%:*} --pidfile "$TMPFILE" --retry 30
		check_retval
	else
		log_success_msg "$DESC: not running"
	fi
	[ -f $PIDFILE ] && rm -f $PIDFILE
}
 
case "$1" in
	start)
		start_bin
	;;
	stop)
		stop_bin
	;;
	force-reload|restart)
		stop_bin
		start_bin
	;;
	status)
		load_settings || exit 4
		if is_running; then
			log_success_msg "$DESC: running (pid $PID)"
		else
			log_success_msg "$DESC: not running"
			[ -f $PIDFILE ] && exit 1 || exit 3
		fi
	;;
	*)
		log_failure_msg "Usage: $0 {start|stop|restart|force-reload|status}"
		exit 3
	;;
esac
 
exit 0

 

And for upstart we have two flavours:

 

The “run a simple binary or script” type:

/weetabytes/blob/master/upstart.conf
# <name of daemon>
#
#
# <Longer description>
#
#
 
description "Derp"
author "Herp Herp D Derp"
 
# The below "start on" line controls when it starts up. There are many options available.
start on (local-filesystems and net-device-up)
#Ditto for stop on.
stop on runlevel [016]
 
respawn
respawn limit 5 30
 
env uid=<user to run as>
env gid=<group to run as>
env umask=000
 
# This next line gets called on start, and killed on stop. Upstart tracks the running invocation automatically.
# I use the below exec line, but you can use just about anything.
 
exec start-stop-daemon -S -c $uid:$gid -k $umask -x <path/to/executable/> -- -d -l <log/me/here.log> -L warning
#exec /usr/bin/binary

 

And the “run some checks and scripting, before running the daemon/binary” type:

/weetabytes/blob/master/upstartscripted.conf
# <name of daemon>
#
#
# <Longer description>
#
#
 
description "Derp"
author "Herp Herp D Derp"
 
# The below "start on" line controls when it starts up. There are many options available.
start on (local-filesystems and net-device-up)
#Ditto for stop on.
stop on runlevel [016]
 
respawn
respawn limit 5 30
 
env uid=<user to run as>
env gid=<group to run as>
env umask=000
#A more insane start condition. This one is based off starting lightdm
start on ((filesystem
           and runlevel [!06]
           and started dbus
           and (drm-device-added card0 PRIMARY_DEVICE_FOR_DISPLAY=1
                or stopped udev-fallback-graphics))
          or runlevel PREVLEVEL=S)
 
stop on runlevel [016]
 
 
#The below lines are run as a shell script on start, and the started binaries killed on stop.
script
	if [ "$RUNLEVEL" = S -o "$RUNLEVEL" = 1 ]
	then
	    # Single-user mode
	    exit 0
	fi
    exec <some binary>
end script
 
#This runs after stop
post-stop script
    rm /some/tmp/files/
end script

 

Now, before, I mentioned some common problems with these. Permissions and environmental options are often the place to start. Using start-stop-daemon you have user selection with --chuid and --chgid. This works with both init and Upstart.

Typically, the simplest troubleshooting step is to run each line of the script, or if you suspect your flow control (IF, ELSE, etc), stuff the lines up in a script of their own, with echo output and sleeps to identify and temporally separate steps.

Sometimes, calling an external bash script can be useful as well. The binaries spawned by that script will still be watched by Upstart for the purpose of stopping them. With init, this is more difficult.

One last thing to remember, both init and Upstart scripts are interpreted as sh, or shell, not bash. Look for bashisms in your script, and if you cannot live without them, call an external script, ensuring it is executed by bash. (A bashism is a procedure or command specific to bash, and not found in other shells, such as sh and ksh.)

See you next week.

-weetabix

Be the first to comment

I’m a git and I love it

Posted March 25, 2013 By weetabix

I use git, and I love it. I use it to hold my personal code and textual files, and I use it to share the code I use on this site, and the code pasted in some of the articles. There is, as always, a lot of contention over svn vs git vs mercurial… but that’s not the point of this article. (Though, I confess, mercurial being written in python tells me I might enjoy fiddling with it.)

The point of this article is, I love git. Most of the time, I don’t even realize I need a repository until … well until I need one. I’m a big fan of bare repos, so thats what I’ll write about. Also, I’ll talk about multiple remote repositories, which is how I manage to keep my pre-published code separate from the github repo I have linked with this blog.

I have three1 github2 plugins3 for WordPress to supply the code to my blog, and I use wp-syntax (or try this for instructions) to colour it up using the fantastic GeSHi highlighting engine and display it.

Git is in all the major distros, and most package managers have a “git” you can install, plus you can find instructions for installing from source, so if you’re a masochist, you can install it on your droid or mini Linux box. There are also a lot of options for Windows. (You can also install git via git…..)

 

So, once you install git, you want a new repo, right? Well, usually, if you are like me, you want a bucket to hold your stuff, something you can organize, push and pull from multiple locations.

* Most of my notes are from Pro Git by Scott Chacon, the git book freely available on the git-scm website.

Often, online, you see repos being set up ending in .git. This is a convention I do not follow.

For a bare repo, go to or mkdir the empty directory, and run


git init --bare

 

Then, you can move in the files you need, or at least touch a file.

The usual commands are:

 

git pushgit commit (or frequently git commit -a),  git pull


man git and git --help are also of some use.

 

Now, once you get the push-pull working for ya, there’s some additional tricks, like your config file in /.git/config. I keep my repo on a remote server from my home, so I have both that repo, and the one I use on github to share code on here.

 

Mine looks like:

/weetabytes/blob/master/git_config
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
[core]
	repositoryformatversion = 0
	filemode = true
	bare = false
	logallrefupdates = true
[remote "origin"]
	fetch = +refs/heads/*:refs/remotes/origin/*
	url = ssh://<user>@<myserver>/repos/git/weetabytes
[branch "master"]
	remote = origin
	merge = refs/heads/master
[remote "github"]
	fetch = +refs/heads/*:refs/remotes/origin/*
	url = ssh://git@github.com/weetabix/weetabytes.git
[remote "all"]
	url = ssh://<user>@<myserver>/repos/git/weetabytes
	url = ssh://git@github.com/weetabix/weetabytes.git

This allows me to do fun stuff, like, after a commit, doing git push github, or git push master, if I only want to update one place, I can do git push all to push to all repos, or I can do git push origin github to make sure both remote repos are up to speed with each other, without my copy being included.

If you are sharing to mostly Linux users, SSH is awesome to use. If you have a large amount of Windows users, http/s allows you to get away without using another program like plink.

Talk to you later,

 

-weetabix

Be the first to comment