Pages

Friday, February 15, 2013

Installing hadoop-1.0.4 on Mac (OS-10.7) for a Single Node



Using the steps given in this post and this excellent guide by Brandon Werner, you can install hadoop-1.0.4 (latest by now) on your mac( iOS-10.7). I will be just adding/modifying some steps (mentioned with the same title) which troubled me to google for those issues. You better follow the other link and when you face some trouble, come back here and see the solution.
  
Getting Java

<same> 

Setting up Environment
 
.bash_profile file was not present under my home directory, so I had to create a new file and edit it. So use following commands to create and modify that file. 

$ touch .bash_profile
$ open -a TextEdit.app .bash_profile

 <rest is same as given in other link>

Downloading Hadoop 

Download 1.0.4.tar.gz file from Apache website and type

$ sudo cp ~/Downloads/hadoop-1.0.4.tar.gz /usr/local/hadoop-1.0.4.tar.gz

$ sudo tar -xvzf hadoop-1.0.4.tar.gz

$ sudo mv ./hadoop-1.0.4 ./hadoop

Configuring: hadoop-env.sh  

Directory name is changed from 'config' to 'conf' in 1.0.4. Rest just follow the steps as given and make a fix for Lion iOS.

Configuring: *-site.xml files  

Remember to open all the files using "sudo vi ..."  and texteditor/root user/sudo chown wont work for this purpose.
 
Setup HDFS for the first time

Make sure 'Remote Login' is enabled in System Preferences->Sharing->Remote Login on your system otherwise you will get an error that Port-22 is closed.

Rest format your HDFS system with the steps given in that link and you better do with #root login.

Startup Hadoop with the Included Scripts

Run start-all.sh script as given. If you see some exception, log in as root and do the following.

# cd /usr/local/hadoop

# bin/stop-all.sh
# bin/hadoop namenode -format
# bin/start-all.sh
If it asks for the password during starting of daemons, given root password.
Rest you can run WordCount example as given in that link.
 Happy hadooping =)

Monday, February 11, 2013

Dynamic analysis of Android APKs using DroidBox


In this tutorial, I will go through installation and usage of dynamic analysis tool (DroidBox) for Android APKs. If you face some problems during installation, you can try a couple of things given in Troubleshooting section.

Step 1: Build Environment

I have installed Ubuntu 12.04 (32 bit) on Oracle VM Virtualbox. And according to Droidbox website, it can only be run on Linux and Mac, but not on Windows. And don't forget to install other dependencies for android SDK and Numpy/Scipy given below.

If you have not installed android SDK, follow the instructions here to get everything installed. Get API level 7 (2.1) installed for this tutorial. My installations are as per the settings (directory locations etc.) given in that link.

Step 2:

Open terminal and export the path for the SDK tools, if you have not followed the link for sdk installation.
 
export PATH=$PATH:/<path/to/android-sdk>/tools/
export PATH=$PATH:/<path/to/android-sdk>/platform-tools/


Step 3:

Change directory to Downloads and download Droidbox.tar.gz file.

$ cd Downloads
$ wget http://droidbox.googlecode.com/files/DroidBox.tar.gz

Step 4:

If you are using Ubuntu 12.04 (32 bit), you can use following commands otherwise, go to the website and install relevant binaries for Numpy and Scipy.

If git is not installed,

$ sudo apt-get install git

$ git clone git://github.com/numpy/numpy.git numpy

$ git clone git://github.com/scipy/scipy.git scipy

Step 5:

During installation, I got an error like I need to install python-dev, so

$sudo apt-get install python-dev

Now we need to install numpy and scipy.

$ cd /Downloads/numpy
$ python setup.py install --user

Let it run and check at the end that you don't get any error.

$ cd /Downloads/scipy
$ python setup.py install --user

Step 6:

Now install MatPlotLib

$sudo apt-get install python-matplotlib 

Step 7:

Now if you have installed SDK as given in the link, you need to keep a few things in mind.

You need to run eclipse with the root user, not with sudo. And after fresh installation, you wont be knowing root password. So lets first change password first and then login with the root to run eclipse.

$ sudo passwd root

and choose password for root. Now login with root

$ su root

after logging in, 

# cd /usr/local/eclipse
#  ./eclipse

Step 8:

Now create an AVD and if you face some error like

'PANIC: cant load AVD etc' or 'Failed to load libGL.so' etc,

Run following command

# apt-get install libgl1-mesa-dev

Command Line AVD

However, if you want to create an AVD by command line, follow instructions given here.
  
At the end of this step, I assume, you have already created an AVD for API Level 7 with name 'test'. And you dont need to start emulator now.

AVD Location

Here are two things now. If you have created an AVD through Eclipse, AVD will be stored by default under root user because you ran it with the root user. Your created AVD will be at /root/.android/AVD/ .

If you created an AVD with command line and normal user, AVD will be stored under /home/mohsin/.android/AVD/

Step 9: Path Settings

 Now Droidbox website says, you download and extract Droidbox anywhere, but I think, they should clearly talk about path settings in the shell scripts also. This missing information definitely took my hours.

Anyway, open startemu.sh file in Droidbox directory, give path to the emulator like below

/opt/android-sdk-linux/tools/emulator -avd $1 -system images/system.img -ramdisk images/ramdisk.img -kernel images/zImage -prop dalvik.vm.execution-mode=int:portable &


Open droidbox.py and give paths to monkeyrunner and adb so that they finally look like following:

call(['/opt/android-sdk-linux/
tools/monkeyrunner', 'scripts/monkeyrunner.py', apkName, runPackage, runActivity], stderr=PIPE)


call(['/opt/android-sdk-linux/platform-tools/adb', 'logcat', '-c'])

Step 10: 

Now it's party time. Lets run emulator and install APK for analysis. Make sure you are logged in as a root if you have created an avd with the emulator.

./startemu.sh test
  
It will automatically get path of test.avd.

9)  Get your APK and give it to droidbox for analysis.

./droidbox.sh <file.apk> <duration in secs (optional)> 
 
e.g.
 
./droidbox.sh ./HippoSMS.apk 100  

If you give 100 as a parameter, it will analyze for 100 seconds otherwise, you will have to do Ctrl+C to stop it.

Here is the output for one of the samples of HippoSMS malware.






--

--

--

--


Thursday, February 7, 2013

Installing Hadoop 1.0.4 on Ubuntu 12.04 (LTS) on Single Node cluster



I have been trying to install Hadoop on Windows using Cygwin but it was not successful because of permissions denied for sshd user. So, I moved on to Ubuntu 12.04 (32 bit) on Oracle VM Virtualbox. 

This whole post is based on installation guide by Michael Noll. So keep opened this post and Noll's installation guide as I will be adding only missing steps given in his post.

Step 1: Sun Java 6

Open a terminal and run following commands to install Sun-java6-dk. Noll's commands didn't work for me.

$ sudo add-apt-repository "deb http://archive.ubuntu.com/ubuntu hardy main multiverse"
$ sudo add-apt-repository "deb http://archive.ubuntu.com/ubuntu hardy-updates main multiverse"
$ sudo add-apt-repository "deb http://archive.canonical.com/ lucid partner"
$ sudo add-apt-repository "deb http://ppa.launchpad.net/webupd8team/java/ubuntu precise main"
$ sudo apt-get update
$ sudo apt-get install sun-java5-jdk sun-java6-jdk oracle-java7-installer

And run following command to confirm jdk6 installation

$ java -version

Step 2:

Add a hadoop system user.

Step 3:

 3.1) :

Configure SSH until you reach following command

hduser@ubuntu:~$ ssh localhost

Above given command didn't work on my machine as it was giving me error like port 22 is closed.
So first of all, we need to add hduser into sudoers list and then install openssh-server using terminal.
So,

3.2)

Login with root or any user which can run sudo commands.

3.3)

$ sudo adduser hduser sudo

$ /usr/sbin/visudo

A file will be opened. Find a line with
root ALL= (ALL:ALL) ALL

Copy paste this line after the root and change 'root' with 'hduser' so that now file will contain two lines like below:
root ALL= (ALL:ALL) ALL
hduser ALL= (ALL:ALL) ALL

3.4)

Now install openssh-server preferably using root login.
# sudo apt-get install openssh-server

3.5) 

Login with hduser account and run following command

$ ssh localhost

and enter 'yes' to continue connecting and complete configuration of ssh.

Step 4: Disable IPV6 

Use following command to open .conf file and gedit was not working on my machine.
$ sudo vim /etc/sysctl.conf

Step 5:   

And for rest of the installation, follow the Nolls' tutorial. In the future, when you are asked to edit a file, use 'sudo vim' for all the files.

Rest of the things went smoothly for me, so I think, Noll's tutorial would suffice for complete installation and get hadoop working for a single node cluster.

Happy hadooping =)


Saturday, February 2, 2013

Fixing MBR error on Windows 7


I had Windows 7 on my 64-bit machine. And then I needed Ubuntu installation in parallel on my computer. I installed Ubuntu 12.04 through Ubuntu-windows installation (wubi.exe) and then I restarted my computer, I faced master boot record (MBR) error. I don't know what happened but somehow it had modified MBR settings. This also happened once when I installed Ubuntu separately and in my Windows, I formatted that disk partition which had Ubuntu installation files. That was foolish of me to do it.

Anyway, here is the quickest trick to get this problem resolved I found on the net.


Step 1: 

Get Ubuntu installed on a CD/DVD. If you don't know how to do it, follow the step1-3 given here.

Step 2: 

Insert your CD/DVD into drive and make sure your machine boots from the CD.

Step 3:

When machine is started, click on Try Ubuntu. Open terminal and install lilo.

$ sudo apt-get install lilo

Step 4:

$ sudo lilo -M /dev/sda mbr

Restart your machine. It should solve MBR problem.

Reference:

http://thameera.wordpress.com/2011/04/24/restoring-the-windows-mbr-using-a-ubuntu-live-cd-or-usb/