RabbIT

Enable filter rabbit.filter.ReverseProxy, by adding it to the front of httpinfilters value.

Configure values for sector rabbit.filter.ReverseProxy.

Run RabbIT:

java -jar jars/rabbit4.jar -f conf/rabbit.conf

Resource: http://www.khelekore.org/rabbit/

Memory allocation settings in Hadoop

Edit file conf/mapred-site.xml to change amount of memory allocated to sorting:

<property> 
    <name>io.sort.mb</name>  
    <value>300</value>  
</property>

Edit file conf/mapred-site.xml to change amount of memory allocated to each map/reduce task:

<property>
    <name>mapred.child.java.opts</name>
    <value>-Xmx800m</value>  
</property>

Edit file conf/hadoop-env.sh to change amount of memory allocated to Hadoop daemons:

export HADOOP_HEAPSIZE=1000

Change ports used by Hadoop

Edit file conf/hdfs-site.xml to change ports used by HDFS

    <property>
        <name>dfs.secondary.http.address</name>
        <value>0.0.0.0:51090</value>
    </property>
    <property>
        <name>dfs.datanode.address</name>
        <value>0.0.0.0:51010</value>
    </property>
    <property>
        <name>dfs.datanode.http.address</name>
        <value>0.0.0.0:51075</value>
    </property>
    <property>
        <name>dfs.datanode.https.address</name>
        <value>0.0.0.0:51475</value>
    </property>
    <property>
        <name>dfs.datanode.ipc.address</name>
        <value>0.0.0.0:51020</value>
    </property>
    <property>
        <name>dfs.http.address</name>
        <value>0.0.0.0:51070</value>
    </property>
    <property>
        <name>dfs.https.address</name>
        <value>0.0.0.0:51470</value>
    </property>

Edit file conf/mapred-site.xml o change ports used by MapReduce

    <property>
        <name>mapred.job.tracker.http.address</name>
        <value>0.0.0.0:51030</value>
    </property>

    <property>
        <name>mapred.task.tracker.http.address</name>
        <value>0.0.0.0:51060</value>
    </property>

exclude directories when using GNU tar

tar zvcf name.tar.gz --exclude path/to/dir1 --exclude path/to/dir2 path/to/tar

Note:

1. Do not include a trailing '/' in the path of excluded directories.  Otherwise, it won't work.
2. Put --exclude before the directory/file to be tarred.

Install RPM packages with non-root account

Recently, I need to install RPM to RedHat Linux, but I don't have root access.  I found this post: http://ajaya.name/?p=6353.  However, some commands in the command are not correct or need more clarification.  So I wrote down my experience below.

• create file ~/.rpmmacros and add following line
      %_rpmlock_path lib/rpm/__db.000
• Initialize the database by running command:

rpm --initdb \

    --root /home/<user_name>/rpm-local/ \

    --dbpath /home/<user_name>/rpm-local/lib/rpm

• Check the dependence

rpm --root /home/<user_name>/rpm-local/ \
    --dbpath /home/<user_name>/rpm-local/lib/rpm \
    -ivh package.rpm

• Install package

rpm --root /home/<user_name>/rpm-local \
    --dbpath /home/<user_name>/rpm-local/lib/rpm \ 
    --relocate /usr=/home/<user_name>/rpm-local \
    --nodeps \
    -ivh package.rpm

Resources

• https://bugzilla.redhat.com/show_bug.cgi?id=222194
• http://www.redhat.com/archives/rpm-list/2007-May/msg00037.html
• https://rhn.redhat.com/errata/RHBA-2006-0440.html

Add "Edit with Vim" item to context-menu in Windows for Vim

This post shows how to add "Edit with Vim" and "Tab Edit with Vim" items to context menu (pop out when you right click a file) in Windows.

Run regedit.exe, go to HKEY_LOCAL_MACHINE/SOFTWARE/Classes/*/shell/

1. Create new key "Tab Edit with &Vim" (Right click parent entry -> New -> Key)
2. Create new key "Edit with Vim"
3. Create new key "command" under "Tab Edit with &Vim"
4. Edit the entry with name "(Default)", change its data to
   <vim_dir>\gvim.exe" -p --remote-tab-silent "%1" "%*"
5. Create new key "command" under "Edit with Vim".
6. Edit the entry with name "(Default)", change its data to
   <vim_dir>\gvim.exe "%1"

It seems that latest versions of vim automatically create the registry entry:

• HKEY_LOCAL_MACHINE\SOFTWARE\Vim\Gvim and the data of entry path points to the vim executable.
• HKEY_CLASSES_ROOT\*\shellex\ContextmenuHandlers\gvim. Data of the entry "(Default)" is the CLSID (51EEE242-AD87-11d3-9C1E-0090278BBD99).
  You can delete this entry because we have already added menu item "Edit with Vim".
• HKEY_CLASSES_ROOT\Applications\gvim.exe\shell\edit\command. Data of the entry "(Default)" is "<vim_dir>\gvim.exe" "%1"
• HKEY_CLASSES_ROOT\CLSID\{51EEE242-AD87-11d3-9C1E-0090278BBD99}
  It has a key named InProcServer32. The data of "(Default)" is <vim_dir>\gvimext.dll
• Some other keys

Logging in Hadoop

Hadoop uses log4j via Apache common logging.  The config file is conf/log4j.properties.

Some important variables are set in the command line.  Following is a snippet cut from the whole command line used to launch HDFS name node.

-Dhadoop.root.logger=INFO,DRFA
-Dhadoop.log.dir=/N/u/hdfs/programs/hadoop-0.21.0/bin/../logs
-Dhadoop.log.file=hadoop-hdfs-namenode-b009.log
-Dhadoop.home.dir=/N/u/hdfs/programs/hadoop-0.21.0/bin/..
-Dhadoop.id.str=hdfs

You can see that log dir, log file, log level, logger are set.  DRFA is defined in conf/log4j.properties.

pdf to eps conversion

Currently, I need to convert pdf files to eps so that they can be included in latex files.

1. Use Acrobat Pro open the pdf file. 
   Click File -> export -> PostScript -> Encapsulated PostScript or use "Save As" and change "Save As Type"
   However, the bounding box is NOT correctly calculated.
   You can use gsview to correct it.  Use gsview open the eps file, click "File -> PS to EPS", select "Automatically calculate Bounding Box" and save the output file.

2. Use ghostscript.  Execute following command:
   gswin32 -sDEVICE=epswrite -sOutputFile=<filename>.eps <filename>.pdf
   This works well and bounding box is correctly calculated.

3. Use Xpdf (http://www.foolabs.com/xpdf/download.html)
   pdftops -eps <filename>.pdf <filename>.eps
   However, the bounding box is NOT correctly calculated.

Brief Latex notes for equations

In-line: $…$

Single line, without equation number: \[ … \] or \begin{equation*} … \end{equation*}

Single line, with equation number: \begin{equation} … \end{equation}

Multi-line, without equation number: \begin{align*} ... \end{align*}

Multi-line, with equation number: \begin{align} ... \end{align}

"a double backslash (\\) is used to separate the lines, and an ampersand symbol (&) is used to indicate the place at which the formulas should be aligned."

For align, \label must be put in front of each equation. For equation, it does not matter you put it in front or in the end.

Make vim-latex to generate output in a specified directory

Recently I started to use latex to write papers.  I want to edit latex in my favorite editor - vim. I found the project vim-latex: http://vim-latex.sourceforge.net/.  It is powerful and convenient to use. 

However, one feature I want is to generate output files (.div, .ps, .log, etc) into a separate directory rather than the same directory as tex files.  It turns out that vim-latex does not support it natively.  So I hacked into the source code to make it work on Windows.

1. Edit file ~/vimfiles/ftplugin/tex.vim

   Add following config:

     set iskeyword+=: 
     let g:Tex_Outdir='out' 
     let g:Tex_ViewRule_pdf='"Foxit Reader.exe" ' 
     let g:Tex_CompileRule_dvi='mkdir '.g:Tex_Outdir.' & latex -output-directory='.g:Tex_Outdir.' -src-specials --interaction=nonstopmode $*' 
     let g:Tex_CompileRule_pdf=g:Tex_CompileRule_dvi.' & cd '.g:Tex_Outdir.' & dvipdfm $*.dvi'

   Basically above config specifies

   1. output directory of latex compilation
   2. How to view PDF files (if you don't give full path, the command needs to be in env variable PATH)
   3. How to compile dvi: create the output directory and put output there
   4. How to compile pdf: first compile tex to dvi, and then call dvipdfm to generate pdf

2. Change ~/vimfiles/ftplugin/latex-suite/compiler.vim

   Change line 252 to (this line adds the full path of output directory):
     let execString = 'start '.s:viewer.' "'.expand('%:p:h').'/'.g:Tex_Outdir.'/$*.'.s:target.'"'

   Change line 405 to (this line adds the full path of output directory):
   let execString = 'silent! !'.viewer.' "'.expand('%:p:h').'/'.g:Tex_Outdir.'/'.mainfnameRoot.'.'.s:target.'" '.line('.').' "'.expand('%').'"'

   Note: the text in blue is what I added (diff against the original code)

Use command 'TTarget' to switch amont dvi, ps, pdf, etc.