Stop Mac OS X creating hidden files when using the tar command

Last week I found something quite frustrating with the tar command on Mac OS X, it likes to put hidden files into archives when you tar them up, it doesn’t give you any warning, just does it.

Creating a tar, and then having a look at its contents, you’ll see something like this :

JamesMac:staging-area JElsey$ tar -tf MyApplication.tar.gz 
src/
src/._MyApplication.cmd
src/MyApplication.cmd
src/._MyApplication.properties
src/MyApplication.properties
src/lib/
src/lib/._anExternalJar.jar
src/lib/anExternalJar.jar

Notice the files prefixed with “._”.

You can quite easily stop this, by setting the following environment variable (I prefer to set this up in the bashrc_profile):

COPYFILE_DISABLE=1; export COPYFILE_DISABLE

Then, tar up the files again, and you should see those hidden files no longer.

JamesMac:staging-area JElsey$ tar -tf MyApplication.tar.gz 
src/
src/MyApplication.cmd
src/MyApplication.properties
src/lib/
src/lib/_anExternalJar.jar

Failing that, you could also install GNU tar instead of the Mac version.

Recursively deleting .svn directories

Messing around with your subversion directories and fed up of the .svn hidden folders laying around? If you try and checkin some directories that contain a .svn folder from some other repository, you’re going to have a whole world of pain trying to fix it (speaking from first hand experience here)

The easiest way to clear up rogue .svn directories is to run this command (on linux or mac). It will recursively find all directories named .svn, and pass them in for removal.

rm -rf `find . -type d -name .svn`

On a Windows 7 environment, I find the easiest way is to just open Windows Explorer in the root directory that you’re interested in, and search for .svn in the top right hand search bar, then highlight all results for .svn and delete, easy :)

Hope this helps

Using special characters in a linux sed command without having to escaping them

I came across a brilliant little feature of the linux sed command today that I wasn’t aware of, and thought it was well worth posting up about.

I’d been using sed statements inside a bash script, to trawl through property files and replace values, these were all alphanumeric values so I didn’t have any problems with the usual sed comand of

sed -e s/replaceThisValue/withThisValue/g

That worked great, but then I had some properties introduced that contained special characters, notably the forward slash and equals. I scanned the interwebz for answers and there were all sorts of complicated techniques for escaping special characters, passing them into awk and all sorts of other such nonsense, however there is a much simpler way, just use a different separator character.

Put some data into a file…

# this will echo the URL and save it into urls.txt
echo http://www.google.com >> urls.txt

Then replace it with the sed command…

# This will replace http://www.google.com with http://www.amazon.com
# The bit after the sed is basically a mechanism for updating the original txt file via a tmp file.
sed -e 's=http://www.google.com=http://www.amazon.com=g' urls.txt > urls.tmp && mv urls.tmp urls.txt

Since the old/new values above contain forward slashes, I didn’t want to bother escaping them, so I’ve just used an equals character instead of the forward slash to denote the segments of the sed command.

This also works with the at symbol, it may work for others too, but I haven’t tried

# The @ symbol works too!
sed -e 's@http://www.google.com@http://www.amazon.com@g' urls.txt > urls.tmp && mv urls.tmp urls.txt

Hope this helps!

How to keep checking if port is open, by issuing socket connects

I recently need to check if a particular application is online and listening on a particular set of ports. I figured that the best way to do this was to periodically issue some socket connects onto those ports. If I got an exception, then clearly there would be an issue.

This was all running on a CentOS Linux machine, so I opted for a Python script, have a look at my sample below

# Socket Connect Script
# Author : James Elsey
# Date : 27 January 2011

# Imports
import socket
import sys
import time
import datetime

# Configure these as required..
remote_host = "127.0.0.1"

print "About to start running script..."

# run continuously
while True:
        time.sleep(60)
        for remote_port in [80,8080]:
                # Setup socket
                sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
                sock.settimeout(60)
                now = datetime.datetime.now()
                now_text = now.strftime("%Y-%m-%d %H:%M:%S")

                try:
                        # Issue the socket connect on the host:port
                        sock.connect((remote_host, remote_port))
                except Exception,e:
                        print "%s %d closed - Exception thrown when attempting to connect. " % (now_text, remote_port)
                else:
                        print  "%s %d open" % (now_text, remote_port)
                sock.close()

So there you have it, take it and modify as you please! You could remove the while True and setup a for loop to check for a specific number of times, or leave the continual loop in and create another method of breaking out, for me I just wanted to leave it running in a putty window and watch for changes.

Also, to run it, just use save it into a .py file and call “python myFile.py”

Happy coding