Sunday, January 16, 2011
Monday, January 03, 2011
Monday, December 06, 2010
Thursday, November 04, 2010
Notes on installing Condor on Windows / Win 7
Below you can read about some of the problems I encountered while testing Condor, along with some solutions/workarounds.
1) The Condor keyboard daemon (KBDD) does not work out of the box. If you start the condor_kbdd.exe process as the SYSTEM user (the default configuration), it will fail to register keyboard/mouse activity. Instead, you can configure Condor to not start the KBDD daemon by removing “KBDD” from the DAEMON_LIST in the configuration file. If you want to use the kbdd process you will need to run it as the the local (logged in) user.
2) When installing Condor manually you need to set the CONDOR_CONFIG environment variable *system wide* (not for the current user). Attempting to start the service when the environment variable is not configured yields this error: “Error 1067: The program terminated unexpectedly”. The manual mentions that instead of an environment variable we can also set several registry keys. I found this to be incorrect. Condor seems to completely ignore those registry keys. If I remember correctly, the installer does not even create them.
3) If the Condor service is configured to start at boot time it will sometimes (always?) fail. One theory is that this happens because Condor tries to establish network connections before the networking services are up. So you might want to delay starting the service until the machine booted up properly.
Below you can read about some of the problems I encountered while testing Condor, along with some solutions/workarounds.
1) The Condor keyboard daemon (KBDD) does not work out of the box. If you start the condor_kbdd.exe process as the SYSTEM user (the default configuration), it will fail to register keyboard/mouse activity. Instead, you can configure Condor to not start the KBDD daemon by removing “KBDD” from the DAEMON_LIST in the configuration file. If you want to use the kbdd process you will need to run it as the the local (logged in) user.
2) When installing Condor manually you need to set the CONDOR_CONFIG environment variable *system wide* (not for the current user). Attempting to start the service when the environment variable is not configured yields this error: “Error 1067: The program terminated unexpectedly”. The manual mentions that instead of an environment variable we can also set several registry keys. I found this to be incorrect. Condor seems to completely ignore those registry keys. If I remember correctly, the installer does not even create them.
3) If the Condor service is configured to start at boot time it will sometimes (always?) fail. One theory is that this happens because Condor tries to establish network connections before the networking services are up. So you might want to delay starting the service until the machine booted up properly.
Sunday, September 19, 2010
Thursday, December 03, 2009
Tuesday, July 07, 2009
Sunday, July 05, 2009
Friday, June 05, 2009
Wednesday, May 20, 2009
Distributed key-value stores
http://www.metabrew.com/article/anti-rdbms-a-list-of-distributed-key-value-stores/ - Anti-RDBMS: A list of distributed key-value stores
http://randomfoo.net/2009/04/20/some-notes-on-distributed-key-stores - Some Notes on Distributed Key Stores
http://bret.appspot.com/entry/how-friendfeed-uses-mysql - How FriendFeed uses MySQL to store schema-less data
http://anyall.org/blog/2009/04/performance-comparison-keyvalue-stores-for-language-model-counts/ - Performance comparison: key/value stores for language model counts
http://michalfrackowiak.com/blog:redis-performance - Redis Performance on EC2 (aka weekend project coming)
http://labs.gree.jp/Top/OpenSource/Flare-en.html - Flare is distributed, and persistent key-value storage - pluggable storage (currently only Tokyo Cabinet is available, though:)
http://blip.tv/file/1949416/ - Drop ACID and think about data
http://www.igvita.com/2009/02/13/tokyo-cabinet-beyond-key-value-store/ - Tokyo Cabinet: Beyond Key-Value Store
http://www.scribd.com/doc/12016121/Tokyo-Cabinet-and-Tokyo-Tyrant-Presentation - Tokyo Cabinet and Tokyo Tyrant Presentation
http://www.metabrew.com/article/anti-rdbms-a-list-of-distributed-key-value-stores/ - Anti-RDBMS: A list of distributed key-value stores
http://randomfoo.net/2009/04/20/some-notes-on-distributed-key-stores - Some Notes on Distributed Key Stores
http://bret.appspot.com/entry/how-friendfeed-uses-mysql - How FriendFeed uses MySQL to store schema-less data
http://anyall.org/blog/2009/04/performance-comparison-keyvalue-stores-for-language-model-counts/ - Performance comparison: key/value stores for language model counts
http://michalfrackowiak.com/blog:redis-performance - Redis Performance on EC2 (aka weekend project coming)
http://labs.gree.jp/Top/OpenSource/Flare-en.html - Flare is distributed, and persistent key-value storage - pluggable storage (currently only Tokyo Cabinet is available, though:)
http://blip.tv/file/1949416/ - Drop ACID and think about data
http://www.igvita.com/2009/02/13/tokyo-cabinet-beyond-key-value-store/ - Tokyo Cabinet: Beyond Key-Value Store
http://www.scribd.com/doc/12016121/Tokyo-Cabinet-and-Tokyo-Tyrant-Presentation - Tokyo Cabinet and Tokyo Tyrant Presentation
Installing Tokyo Tyrant on Ubuntu 9.04 (Jaunty), in custom folders/directories
Tokyo Cabinet - first install this
-------------
cd /usr/local/src/
wget http://tokyocabinet.sourceforge.net/tokyocabinet-1.4.21.tar.gz
tar xvf tokyocabinet-1.4.21.tar.gz
cd tokyocabinet-1.4.21
apt-get install zlib1g-dev
apt-get install libbz2-dev
mkdir /usr/local/tokyocabinet-1.4.21/
./configure --prefix=/usr/local/tokyocabinet-1.4.21/
make
make install
Tokyo Tyrant
------------
cd /usr/local/src/
wget http://tokyocabinet.sourceforge.net/tyrantpkg/tokyotyrant-1.1.27.tar.gz
tar xvf tokyotyrant-1.1.27.tar.gz
cd tokyotyrant-1.1.27
mkdir /usr/local/tokyotyrant-1.1.27
./configure --prefix=/usr/local/tokyotyrant-1.1.27/ --with-tc=/usr/local/tokyocabinet-1.4.21/
make
make install
Fix lib problem
------------
cd /usr/local/tokyotyrant-1.1.27/bin/
./ttserver
You will get this error: ./ttserver: error while loading shared libraries: libtokyocabinet.so.8: cannot open shared object file: No such file or directory
To fix it,
cd /usr/local/tokyotyrant-1.1.27/
ln -s /usr/local/tokyocabinet-1.4.21/lib/libtokyocabinet.so.8 lib/
./bin/ttserver will now work
Tokyo Cabinet - first install this
-------------
cd /usr/local/src/
wget http://tokyocabinet.sourceforge.net/tokyocabinet-1.4.21.tar.gz
tar xvf tokyocabinet-1.4.21.tar.gz
cd tokyocabinet-1.4.21
apt-get install zlib1g-dev
apt-get install libbz2-dev
mkdir /usr/local/tokyocabinet-1.4.21/
./configure --prefix=/usr/local/tokyocabinet-1.4.21/
make
make install
Tokyo Tyrant
------------
cd /usr/local/src/
wget http://tokyocabinet.sourceforge.net/tyrantpkg/tokyotyrant-1.1.27.tar.gz
tar xvf tokyotyrant-1.1.27.tar.gz
cd tokyotyrant-1.1.27
mkdir /usr/local/tokyotyrant-1.1.27
./configure --prefix=/usr/local/tokyotyrant-1.1.27/ --with-tc=/usr/local/tokyocabinet-1.4.21/
make
make install
Fix lib problem
------------
cd /usr/local/tokyotyrant-1.1.27/bin/
./ttserver
You will get this error: ./ttserver: error while loading shared libraries: libtokyocabinet.so.8: cannot open shared object file: No such file or directory
To fix it,
cd /usr/local/tokyotyrant-1.1.27/
ln -s /usr/local/tokyocabinet-1.4.21/lib/libtokyocabinet.so.8 lib/
./bin/ttserver will now work
Tuesday, March 24, 2009
Thursday, March 12, 2009
Monday, February 02, 2009
Pipe Viewer
"Pipe viewer is a terminal-based tool for monitoring the progress of data through a pipeline. It can be inserted into any normal pipeline between two processes to give a visual indication of how quickly data is passing through, how long it has taken, how near to completion it is, and an estimate of how long it will be until completion."
http://www.catonmat.net/blog/unix-utilities-pipe-viewer/
"Pipe viewer is a terminal-based tool for monitoring the progress of data through a pipeline. It can be inserted into any normal pipeline between two processes to give a visual indication of how quickly data is passing through, how long it has taken, how near to completion it is, and an estimate of how long it will be until completion."
http://www.catonmat.net/blog/unix-utilities-pipe-viewer/
Tuesday, January 06, 2009
Protecting from a DDOS SYN FLOOD on port 80 (Apache, Ubuntu)
Other keywords: SYN_RECV, TIME_WAIT
sysctl -w net.ipv4.tcp_synack_retries="1"
sysctl -w net.ipv4.tcp_max_syn_backlog="40000"
sysctl -w net.ipv4.netfilter.ip_conntrack_max="200000"
sysctl -w net.ipv4.tcp_fin_timeout="3"
/sbin/iptables -A INPUT -i eth2 -p tcp --tcp-flags ALL ACK,RST,SYN,FIN -j DROP
/sbin/iptables -A INPUT -i eth2 -p tcp --tcp-flags SYN,FIN SYN,FIN -j DROP
/sbin/iptables -A INPUT -i eth2 -p tcp --tcp-flags SYN,RST SYN,RST -j DROP
(you can ignore the iptables part, probably doesn't help that much)
In Apache, increase the maximum number of concurrent connections.
I also learned that while in theory putting a squid in reverse proxy on port 80 to protect the Apache behind it is a good idea, squid sucks at it.
Oh, I almost forgot. Make sure syn_cookies is active (if you kernel supports it):
echo 1 > /proc/sys/net/ipv4/tcp_syncookies
Google is your friend for further details on the above settings.
More:
http://tools.ietf.org/html/rfc4987#section-3.5
http://www.cisco.com/web/about/ac123/ac147/archived_issues/ipj_9-4/syn_flooding_attacks.html
Other keywords: SYN_RECV, TIME_WAIT
sysctl -w net.ipv4.tcp_synack_retries="1"
sysctl -w net.ipv4.tcp_max_syn_backlog="40000"
sysctl -w net.ipv4.netfilter.ip_conntrack_max="200000"
sysctl -w net.ipv4.tcp_fin_timeout="3"
/sbin/iptables -A INPUT -i eth2 -p tcp --tcp-flags ALL ACK,RST,SYN,FIN -j DROP
/sbin/iptables -A INPUT -i eth2 -p tcp --tcp-flags SYN,FIN SYN,FIN -j DROP
/sbin/iptables -A INPUT -i eth2 -p tcp --tcp-flags SYN,RST SYN,RST -j DROP
(you can ignore the iptables part, probably doesn't help that much)
In Apache, increase the maximum number of concurrent connections.
I also learned that while in theory putting a squid in reverse proxy on port 80 to protect the Apache behind it is a good idea, squid sucks at it.
Oh, I almost forgot. Make sure syn_cookies is active (if you kernel supports it):
echo 1 > /proc/sys/net/ipv4/tcp_syncookies
Google is your friend for further details on the above settings.
More:
http://tools.ietf.org/html/rfc4987#section-3.5
http://www.cisco.com/web/about/ac123/ac147/archived_issues/ipj_9-4/syn_flooding_attacks.html
Wednesday, November 26, 2008
Monday, October 06, 2008
Linux/Ubuntu Shift keyboard bug when using VMware Player or Workstation
Symptom: "your keyboard outside the vm client stops working correctly no capslock, shift nor control is working" - https://bugs.launchpad.net/ubuntu/+bug/187165 Same bug here also: https://bugs.launchpad.net/ubuntu/+source/xorg/+bug/195982
Fix: Run the command setxkbmap in a terminal window
Update: Seems to be fixed in VMware Workstation 6?
Symptom: "your keyboard outside the vm client stops working correctly no capslock, shift nor control is working" - https://bugs.launchpad.net/ubuntu/+bug/187165 Same bug here also: https://bugs.launchpad.net/ubuntu/+source/xorg/+bug/195982
Fix: Run the command setxkbmap in a terminal window
Update: Seems to be fixed in VMware Workstation 6?
Tuesday, September 16, 2008
Subscribe to:
Posts (Atom)