Keli's Mumbo Jumbo

To install blpapi library for python on Mac OS, first download the C++ sdk from bloomberg.

Then unpack the tarball to a path you prefer. Take /usr/local for example:

1
2
3
4

cd /usr/local
sudo tar xfv ~/Downloads/blpapi_cpp_3.15.0.1-macos.tar.gz
# create a symlink to make future updates a bit easier
sudo ln -s blpapi_cpp_3.15.0.1 blpapi

Next, try installing blpapi for python with pip, with env variable BLPAPI_ROOT pointing to the C++ sdk's path.

1

BLPAPI_ROOT=/usr/local/blpapi pip install --index-url=https://bloomberg.bintray.com/pip/simple --upgrade blpapi

If you have the lastest xcode CLT installed, you might see a compiler error though:

1
2
3
4

blpapi/internals_wrap.c:20050:19: error: implicit declaration of function 'blpapi_ZfpUtil_getOptionsForLeasedLines' is invalid in C99
      [-Werror,-Wimplicit-function-declaration]
    result = (int)blpapi_ZfpUtil_getOptionsForLeasedLines(arg1,(struct blpapi_TlsOptions const *)arg2,arg3);
                  ^

In this case, you have to fix the error first, goto bintray and download the python source tarball, then unpack it.

1
2
3
4

cd ~/Downloads
wget https://bloomberg.bintray.com/pip/simple/blpapi/blpapi-3.15.2.tar.gz
tar xfv blpapi-3.15.2.tar.gz
cd blpapi-3.15.2

A quick search in the C++ sdk shows that the header file that defines the function in question is not included, just open blpapi/internals_wrap.c in the python source directory and add a line to include the header file will fix the error.

1

#include "blpapi_zfputil.h"

Then proceed to install:

1

python setup.py install

That should be it! For much older versions I had to modify the generated _internal.so using install_name_tool -change to help locate libblpapi3.so, which was not as pleasant an experience. Fortunately the newer versions have fixed the issue. It's worth mentioning that the installation on Linux is basically the same, with only one additional step to help locate libblpapi3.so, either by copying the file to /lib/ or add the path of C++ sdk to ld.so.conf.

Hope this helps.

The most light-weight setup, to have a long-running GUI application on Linux server, is to simply use a vncserver, then run the application after connected to the vncserver. This way you don't have to install many of the X11-related packages, and memory footprint is considerably smaller.

Step 1: Install vncserver and a window manager of your choosing (I use icewm).

1

apt install xterm xfonts-base xfonts-scalable tightvncserver icewm sakura

Where sakura is a light-weight terminal emulator that I prefer.

Step 2: Create init files for vncserver session.

1
2
3
4
5
6
7

mkdir .vnc
cat > /.vnc/xstartup <<EOF
#!/bin/bash
xrdb \$HOME/.Xresources
exec /etc/X11/Xsession
EOF
chmod +x .vnc/xstartup

You could also create a vnc password by running vncpasswd.

Step 3: Run vncserver

1

vncserver :1 -depth 24 -geometry 1280x800 -localhost -interface lo -nolisten tcp

Vncserver will listen on localhost:5901, and you're pretty much done at the server side. You could also make it a systemd service instead:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

sudo tee /etc/systemd/system/vncserver@.service <<EOF
[Unit]
Description=Remote desktop service (VNC)
After=syslog.target network.target

[Service]
Type=forking
User=your_user
Group=your_group
WorkingDirectory=/home/your_user
PIDFile=/home/your_user/.vnc/%H:%i.pid
ExecStartPre=-/usr/bin/vncserver -kill :%i > /dev/null 2>&1
ExecStart=/usr/bin/vncserver :%i -depth 24 -geometry 1280x800 -localhost -interface lo -nolisten tcp
ExecStop=/usr/bin/vncserver -kill :%i

[Install]
WantedBy=multi-user.target
EOF

systemctl daemon-reload
systemctl enable vncserver@1.service
systemctl start vncserver@1.service

To connect to the vncserver, you'll need to forward the vnc connection to localhost:5091 on the server side via a ssh tunnel:

1

ssh -L 5901:localhost:5901 -C -N server_ip

On Mac OS, I recommend Secure Pipes which is a great tool for managing multiple tunnels and keeping connections alive.

Now use any vnc client and connect to localhost:5901, you'll be connected to the vncserver. On Mac OS, simply launch Screen Sharing from spotlight, it's a fairly good vnc client.

Quick note really:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

# install nginx, mrtg, and fcgiwrap to wrap mrtg cgi since nginx doesn't support cgi directly
sudo apt-get install nginx mrtg mrtg-rrd rrdtool librrds-perl fcgiwrap

# prepare directories
sudo mkdir -p /var/www/mrtg /var/log/mrtg
sudo chown -R www-data.  /var/www/mrtg /var/log/mrtg
sudo -u www-data cp /usr/share/mrtg/* /var/www/mrtg# prepare the cron job, 1 minute interval is used
sudo sh -c 'echo "* * * * * www-data if [ -x /usr/bin/mrtg ] && [ -r /etc/mrtg.cfg ]; then mkdir -p /var/log/mrtg ; env LANG=C /usr/bin/mrtg /etc/mrtg.cfg 2>&1 | tee -a /var/log/mrtg/mrtg.log ; fi" > /etc/cron.d/mrtg'

# Generate config
sudo sh -c "cfgmaker --global 'Interval: 1' --global 'LogFormat: rrdtool' --global 'IconDir: /mrtg' 10.0.12.254  /etc/mrtg.cfg"
sudo rm -rf /var/lock/mrtg/

### edit /etc/nginx/sites-enabled/default and include /usr/share/doc/fcgiwrap/examples/nginx.conf
sudo service nginx reload

Then visit http://localhost/cgi-bin/mrtg-rrd.cgi/ for the graph

Furion is an encrypted proxy (SOCKS5 + SSL) written in Python. I've written it to help myself get over the Great Firewall of... you know where.

It has no external dependencies other than a standard Python 2.x (>2.5) installation. The advantages of Furion vs a SSH Socks5 proxy are:

• You don't need to reconnect if something happens and your network breaks for a while (e.g. put your laptop to sleep/standby)
• Easier to manage/share accounts
• Better security, it only allows proxy to destination port 80/443, and prevents DNS leak by not allowing using IP address as destination in the protocol.
• Can easily use other Furion server as upstream, making a chain of proxies.

Here is how I use it: I run a copy of Furion on a small VPS box of mine in the US, configured as server (with SSL and authentication turned on), and on my desktop I run Furion configured as client, with SSL and authentication off,  and using my VPS as upstream proxy. This way I can configure my browser to use the local Furion as socks5 proxy and visit any website that my VPS can reach. It's pretty cheap to buy a low-end VPS nowadays (I personally have a few $15/year ones), plus you can share with your friends too once you get it working. Here is a simple diagram showing how it works:

graph TD
Browser--socks5-->FC(Furion Client);
FC--socks5 + ssl-->GFW(The Great Firewall);
GFW-->FS(Furion Server);
FS-->WS(Web Site);
WS-->FS;
FS--socks5 + ssl-->GFW;
GFW-->FC;
FC(Furion Client)--socks5-->Browser;

Firefox and Chrome have extensions that can auto switch to use a proxy when visiting a banned URL (e.g. autoproxy for FireFox and SwitchySharp for Chrome), which make this much easier to use for the purpose.

Code is in a HG repo on bitbucket. No GUI yet, just a plain console program, and all settings are done in configuration file: furion.cfg. There are two cfg examples in the examples directory. There is also an executable for Windows configured to use my VPS as upstream that should be ready for immediate use. Don't abuse this or it might be taken down anytime. Download the windows executable here. By default it listens on 127.0.0.1 port 11080. Set that as socks5 proxy address in your browser and you are good to go.

A few caveats:

1. Many recent versions of Internet Explorer don't have SOCKS5 support at all. Firefox/Chrome are recommended instead, you will want to use one of the automatic proxy switching extensions mentioned above anyway.
2. FireFox users need to set network.proxy.socks_remote_dns to true in about:config, otherwise Furion would refuse connections. This is done by design to avoid DNS poisoning.
3. When running Furion server, you need a pem file for SSL encryption, I have put one in the examples directory for convenience, but keep in mind it's not safe to use that same pam file, generate your own pem file instead if you care.
4. To enable authentication for Furion server, you need a simpleauth.passwd and write username/password in it. An example is also available in the examples directory in source tree.
5. On a OpenVZ VPS with small memory (256M or less), you probably want to set a smaller stack size than default, like "ulimit -s 256", to reduce memory usage, or Furion could have problem when allocating many new threads.

I was trying to register a domain with the .ms extension. 101domain.com was ranked the 1st place in the google search for “.ms domain registration” (turns out 101domain.com does a great SEO job and is ranked the first place for a lot of registrar related search terms). So I registered the domain name I wanted there and paid, then I found out the domain status in my account page was “Sent to registry, pending create”. I thought that .ms might had some restrictions and waited for 4 days. Nothing happened. I then submitted a supported ticket and waited for another 36 hours, again nothing happened, no reply, nothing. So I tried registering the same .ms domain name at another registrar that I read about during my searches, guess what, instant success! I can use the domain name immediately after payment. I couldn’t believe it. Now I’m requesting for a refund from 101domain, see how that goes (I have a bad feeling about this though, probably will have to file a dispute with paypal or even ask my bank to charge back).

In short, 101domain is a scam, avoid at all costs.

Update: They did refund me within 48 hours though, after I filed a dispute via paypal and threatened to ask my bank to charge back.

Earlier today someone asked me to help with installing typical apache/mysql/php combo on a Linux server. After logging in I found that the distro is RHEL4. Obviously yum is not available there... as shitty as yum is, it's still better than the infamous rpm hell, right? So I decided to install yum. After downloading a yum rpm, it failed to install because a bunch of dependencies were not met. "Hell," I started to wonder, "if I have to go through this why not install apt for what it's worth?"

Then I googled a bit and made apt work. In order to install mysql-server and php-mysql I also had to use CentOS's repository. Here is what I did:

• Download & install the apt rpm package from
  http://dag.wieers.com/packages/apt

• (Optional) Download & install the rpmforge package from
  http://dag.wieers.com/packages/rpmforge-release

• Import CentOS 4 PGP key:

  rpm --import http://mirror.centos.org/centos/RPM-GPG-KEY-CentOS-4

• Add CentOS 4 repository to apt sources list by editing /etc/apt/sources.list.d/os.list and uncomment the lines about mirror.centos.org. You can also change the url to a mirror closer to you, mine looks like this:

1
2
3
4

repomd http://ftp.nara.wide.ad.jp/pub/Linux/centos/$(VERSION)/os/$(ARCH)  
repomd http://ftp.nara.wide.ad.jp/pub/Linux/centos/$(VERSION)/updates/$(ARCH)  
repomd http://ftp.nara.wide.ad.jp/pub/Linux/centos/$(VERSION)/extras/$(ARCH)  
repomd http://ftp.nara.wide.ad.jp/pub/Linux/centos/$(VERSION)/fasttrack/$(ARCH)  

if you want to install mysql 5 and php 5 (4 by default), you can also add a line for centosplus:

1

repomd http://ftp.nara.wide.ad.jp/pub/Linux/centos/$(VERSION)/centosplus/$(ARCH)

Now you can use your normal apt commands like apt-get update && apt-get install. Sweeeet!

For third-party repositories you can also use atrpms.net repo.

Sometimes I need to ssh to my servers back at school when working in the company; Since the corporate proxy servers support HTTPS and don't block connections to port 22, ssh over proxy is possible, and here is what I did to make it work:

• Install connect-proxy:

  1	apt-get install connect-proxy

  The debian package description is pretty clear in describing its functions:

  1	Description: Establish TCP connection using SOCKS4/5 and HTTP tunnelconnect-proxy is a simple relaying command to make TCP network connectionvia SOCKS or https proxies.It is mainly intended to be used as proxy command of OpenSSH.

• Edit ~/.ssh/config and add the following lines (modify to suit your needs):

  1 2 3 4 5 6 7 8 9

  # not using proxy on lan Host 172.16.* ProxyCommand connect-proxy %h %p Host localhost ProxyCommand connect-proxy %h %p # mandatory to access the internet Host * ProxyCommand connect-proxy -H proxyserver:proxyport %h %p

Now when you run ssh to connect to servers not on your LAN, connect-proxy is used along transparently.

I played with the following glade example files from LablGtk2:

1. gladecalc.ml
2. project2.glade

You can find them from the liblablgtk2-ocaml-dev package if you are using Debian Gnu/Linux Sid.

To run the gladecalc.ml you should do the following first:

1

lablgladecc2 -embed project2.glade > project2.ml

then you can run gladecalc.ml:

1

lablgtk2 -thread gladecalc.ml

This might not be very useful if you want to deploy your own LablGtk2 applications and the clients don't have OCaml toplevel installed: lablgtk2 is just a shell script wrapped around OCaml toplevel after all.

In our case, to compile gladecalc.ml into a native executable, we should first change the code a little bit. The #use directive in line 6 of gladecalc.ml works only for OCaml toplevel, so comment it out and add a new line:

1
2

(* #use "project2.ml";; *)
open Project2

then use ocamlopt to compile:

1

ocamlopt -I +lablgtk2 lablgtk.cmxa lablglade.cmxa gtkInit.cmx project2.ml gladecalc.ml -o gladecalc

this produces a single executable "gladecalc" without OCaml dependencies.