• |
  • |
  • |
  • |

Welcome!

Be aware that this is not a conventional blog. It will not be updated on a daily basis. It's more of a toolbox with things I encounter during my work or problem resolutions found after some daunting trial and error process. Fruitful reading!

Witaj!

To nie jest typowy blog. Nie będzie aktualizowany codziennie. Jest to raczej przybornik z informacjami, linkami i rozwiązaniami problemów, które napotykam w czasie pracy, często na zasadzie prób i błę­dów. Owocnego czytania!

Bienvenue!

Ce n'est pas un blog typique. Il ne sera pas mis à jour quotidiennement. C'est plutôt une boite à outils avec des choses que je rencontre pendant mon travail ou les solutions des problèmes qui sont à trouver seulement après une recherche exhaustive. Fructueuse lecture!

Willkommen!

Beachten Sie, dass das kein Übliches Blog ist. Ich werde es nicht täglich aktualisieren. Es ist eher ein "Werkzeugkasten" mit den Sachen, die ich während meiner Arbeit antreffe und Problemlösungen, die manchmal nur durch Ausprobieren gefunden werden können. Fruchtbares Lesen!

¡Bienvenido!

Esto no es un blog típico. No voy a actualizarlo diariamente. Es más bien una "caja de herramientas" con las cosas que encuentro durante mi trabajo o resoluciones de los problemas que descubro después de una búsqueda exhaustiva. ¡Lectura provechosa!

Archive Page 3 of 8



USRobotics USR9108 serial console access

I had to debug my USRobotics router recently (a good old USR9108) as it ceased to respond completely to any network traffic. The CFE was working fine (192.168.1.1 could be pinged), so I thought a tftp firmware flash would do the trick, but to no avail (I got timeouts all the time). Accessing the serial console seemed to be the only way left, so I had to practise some PCB soldering the hard way.

A good tutorial would have been really reassuring at the moment. There’s a lot of material on Linksys routers (due to its OpenWrt popularity), but I couldn’t find much about USR9108, so I had to pick and guess the proper pinout. Now I know, doing it with a properly working multimeter (mine is probably appropriate for battery testing only) would have been much easier.

The USR9108 has a 4-pin pad next to the Broadcom BCM6348 CPU. It’s 3.3V transistor-transistor logic (TTL), so a special USB converter (or some reworked USB Nokia data cable) is a must (mine is FT232R based, and works flawlessly). The proper connection sequence is TX, RX, NC, GND as seen below:

Router’s TX and RX have to be connected with RX and TX on the PC side correspondingly, and GND with GND (it’s data/signal ground, so just grounding it is not the best idea). Connection settings which worked in my case are as follows:

Baud rate: 115200
Data bits: 8
Stop bits: 1
Parity control: none
Flow control: software (XON/XOFF)

I was having some problems with RealTerm (though I don’t know why), but PuTTy worked a treat.

More technical details about the router can be found in the OpenWrt wiki.

 

Increasing swap space in Xen VPS using swap files

Problem:

Getting back after some major VPS problems (repercussions of the turmoil CheapVPS was in after the attack) I had to rebuild my system from scratch. Fortunately, my VPS wasn’t directly affected by the attack, so no data loss occurred, but the host node I resided on got crippled so badly it had to be recreated before it could reboot all the domUs. But as always, something must have gone wrong, and my VPS was network dead for almost a week. After some unsuccessful bug squashing, the only way to go was to relocate to a new VPS instance.

As previously, the swap partition was not installed properly. However, this time no swap disk was available and the /etc/fstab trick didn’t work. So I thought it would be faster to sort it out on my own, than file another support ticket after a week of struggling to get my VPS up and running.

Solution:

When there’s no dedicated swap partition, a workaround is possible by the means of swap files. The bad thing about them is they consume the space within allotted disk limits. But the good side is they are always available and their size can be tweaked to one’s liking (or probably needs).

The procedure is quite straightforward, but should be done with caution (as the commands employed are of destructible nature):


1. First, create an empty file which will serve as a swap file by issuing the following command:

dd if=/dev/zero of=/swap bs=1024 count=1048576

where /swap is the desired name of the swap file, and count=1048576 sets the size to 1024 MB swap (for 512 MB use 524288 respectively).


2. Set up a Linux swap area with:

mkswap /swap

Old versions of mkswap needed the size of the swap file/partition, but with newer ones it’s better not to specify it at all since a small typo can erase the whole disk.


3. It’s wise to set the permissions as follows:

chmod 0600 /swap


4. The last thing – add the new swap file to /etc/fstab:

/swap            swap          swap     defaults,noatime           0 0

This way it will be loaded automatically on boot.


5. To enable the new swap space immediately, issue:

swapon -a


And done. Check with free -m if everything went right. You should be seeing additional swap space available. Though it’s an easy way to increase available swap on Xen VPS systems, swap files won’t do for deficient memory. But they provide a nice cushion for temporary spikes – it’s better to serve content slowly, than die completely in the process, locking out the entire system.

Atomic directory symlink replacement with mv ‘no-target-directory’ option

Problem:

This time in one of my scripts I had to create a replicated filesystem-like structure (whole directory tree with selected files) using symbolic links. One of the problems with managing this kind of structure was maintaining an appropriate resolution for all the symlinks throughout the whole update process of the replicated tree, which occurred occasionally. Preferably atomic rename would do the trick. And as Linux ‘mv’ is nothing more than a nice wrapper around rename call, doing this right should not have been hard at all.

In fact, a simple problem occurred not where I would suspect. I couldn’t force the ‘mv’ command to rename a symlink if the destination had already existed as a symbolic link to another directory.

For instance, let’s assume the following structure:
./foo -> ./baz (foo being a symbolic link to directory baz)
./bar -> ./qux (bar being a symbolic link to directory qux)

Then executing:
mv foo bar
would in fact move the foo symlink under qux directory (symlinked as bar).

Though this seemed a bit absurd, I could not find a way to make it work as desired (forcing an overwrite of bar with foo).

Solution:

As always, the most trivial solution was the one I hadn’t thought about. Condemned once again for working on archaic systems, I found out that quite a long time ago (but later than the installation of that particular system) an option precisely dedicated for this problem was added to the ‘mv’ command.

All the recent versions of ‘mv’ support the ‘-T’ or ‘–no-target-directory’ switch. It disables treating the last operand in a special way when it’s a directory/symlink to a directory.

So doing:
mv -f -T foo bar
would work as expected. Though I’m not 100% sure about its atomicity (according to POSIX it should be), it’s the best I could get. And I haven’t noted any problems/race conditions so far.

More about the idea behind ‘Target directory’ behaviour can be found in the GNU Coreutils manuals.

Last but not least, always check if you’re working on the latest version of your tools. Or at least if updating won’t fix your issue right away.

ISPAG – Awesome Oscillator

Poproszono mnie ostatnio o implementację niestandardowego wskaźnika do ISPAG-a (aplet wykresów) spotykanego pod nazwą Awesome Oscillator lub czasami zamiennie Elliott Wave Oscillator.

Pomyślałem, że może przydać się jeszcze komuś, więc go w tym miejscu publikuję:

/* *** ISPAG - wskaźniki użytkownika *************************************** *
 * Awesome Oscillator / Elliott Wave Oscillator                              *
 * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
 * Wersja: 1.0.0 (2009-03-22)                                                *
 * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
 * Opis:                                                                     *
 *    Implementacja  niestandardowego  oscylatora  (Awesome  Oscillator  lub *
 *    inaczej Elliott Wave Oscillator) dla apletu ISPAG 9.5.                 *
 *                                                                           *
 * Domyślne parametry:                                                       *
 *    Param(1) = 5                                                           *
 *    Param(2) = 34                                                          *
 *    Param(3) = 14                                                          *
 *                                                                           *
 * Opis parametrów:                                                          *
 *    Param(1) - liczba okresów szybkiej SMA                                 *
 *    Param(2) - liczba okresów wolnej SMA                                   *
 *    Param(3) - generowanie sygnałów kupna/sprzedaży na podstawie reguł:    *
 *       4 = Saucer                                                          *
 *       8 = Twin Peaks                                                      *
 *       6 = Zero Line + Saucer                                              *
 *       10 = Zero Line + Twin Peaks                                         *
 *       12 = Saucer + Twin Peaks                                            *
 *       14 = Zero Line + Saucer + Twin Peaks                                *
 *                                                                           *
 * Wyjaśnienie sygnałów:                                                     *
 *    http://www.metaquotes.net/techanalysis/indicators/awesome_oscillator   *
 *                                                                           *
 * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
 * Copyright (c) 2009 Division Logiciel                                      *
 * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
 *          Aktualna wersja: http://blog.dvl.pl/snippets/ispag               *
 *        Kontakt z autorem: karol (na) dvl (kropka) pl                      *
 * ************************************************************************* */

/* ************************************************************************* *
 * Changelog:                                                                *
 * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
 * + 2009-03-22 v1.0.0                                                       *
 *   - Pierwsze wydanie                                                      *
 * ************************************************************************* */

/* ************************************************************************* *
 * License: GPL v3 or later (http://www.gnu.org/licenses/gpl.txt)            *
 * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
 * This program is free  software; you can redistribute  it and/or modify it *
 * under the terms  of the GNU  General Public  License as  published by the *
 * Free Software  Foundation; either  version 3 of the  License, or (at your *
 * option) any later version.                                                *
 *                                                                           *
 * This program  is  distributed  in the hope  that it  will be  useful, but *
 * WITHOUT   ANY    WARRANTY;   without   even  the   implied   warranty  of *
 * MERCHANTABILITY or FITNESS FOR A  PARTICULAR PURPOSE. See the GNU General *
 * Public License for more details.                                          *
 *                                                                           *
 * You should have received  a copy of the GNU General  Public License along *
 * with this program. If not, see <http ://www.gnu.org/licenses/>.            *
 * ************************************************************************* */

/* ************************************************************************* *
 * Implementacja wskaźnika:                                                  *
 * ************************************************************************* */

/* Rysowanie wskaźnika - ze względu na brak histogramów w formie ciągłej     */ 

   var high = High();
   var low = Low();

   var median = CreateArray(high.length);

   for (var i=0; i<median .length; i++) {
      median[i] = (high[i] + low[i]) / 2;
   }

   var sma1 = SimpleAvg(median, Param(1));
   var sma2 = SimpleAvg(median, Param(2));

   var AO = CreateArray(median.length);

   for (var i=0; i<AO.length; i++) {
      AO[i] = sma1[i] - sma2[i];
   }

   AddGraph(AO, 0);
   AddHorizLine(0);

/* Generowanie sygnałów kupna/sprzedaży                                      */

   var zeroCross = 0;
   var saucer    = 0;
   var twinPeaks = 0;

   /* Ustalenie pożądanego trybu generowania sygnałów */
   switch (Param(3)) {
      case 2:
         zeroCross = 1;
      break;
      case 4:
         saucer    = 1;
      break;
      case 8:
         twinPeaks = 1;
      break;
      case 6:
         zeroCross = 1;
         saucer    = 1;
      break;
      case 10:
         zeroCross = 1;
         twinPeaks = 1;
      break;
      case 12:
         saucer    = 1;
         twinPeaks = 1;
      break;
      case 14:
         zeroCross = 1;
         saucer    = 1;
         twinPeaks = 1;
      break;
   }

   var lastHigh = 0;
   var lastLow  = 0;
   var momentum = 1;

   /* Generowanie sygnałów, implementacja najprostsza/najczytelniejsza.
    * Przy dużej ilości obrabianych danych mogłaby się przydać optymalizacja
    * w paru miejscach.
    */
   for (var i=0; i<AO.length; i++) {
      if (zeroCross || twinPeaks) {
         /* Zero Cross - przebicie poziomu 0 */
         if (AO[i-1] * AO[i] < 0) {
            if (twinPeaks) {
               /* Twin Peaks - reset */
               lastLow  = 0;
               lastHigh = 0;
               momentum = 1;
               /* Twin Peaks - reset end */
            }
            if (zeroCross) {
               if (AO[i] > 0)
                  AddBuySignal(i);
               else
                  AddSellSignal(i);
            }
         }
      }

      if (saucer) {
         /* Saucer - odwrócenie */
         if (AO[i] > 0 && AO[i-1] > 0 && AO[i-2] > 0) {
            if (AO[i-2] < AO[i-3] && AO[i-1] < AO[i-2] && AO[i] > AO[i-1])
               AddBuySignal(i);
         }
         if (AO[i] < 0 && AO[i-1] < 0 && AO[i-2] < 0) {
            if (AO[i-2] > AO[i-3] && AO[i-1] > AO[i-2] && AO[i] < AO[i-1])
               AddSellSignal(i);
         }
      }

      if (twinPeaks) {
         /* Twin Peaks - mniejszy szczyt */
         if (AO[i] < 0) {
            if (AO[i] > AO[i-1]) {
               if (momentum) {
                  if (AO[i-1] > lastLow) AddBuySignal(i-1);
                  lastLow = AO[i-1];
                  momentum = 0;
               }
            } else {
               momentum = 1;
            }
         }
         if (AO[i] > 0) {
            if (AO[i] < AO[i-1]) {
               if (momentum) {
                  if (AO[i-1] < lastHigh) AddSellSignal(i-1);
                  lastHigh = AO[i-1];
                  momentum = 0;
               }
            } else {
               momentum = 1;
            }
         }
      }
   }
/*::~ EOF */

Co do skuteczności samego wskaźnika nie będę się wypowiadał, bo nie miałem go okazji testować w praktyce.

Z zasadą działania można się zapoznać na następujących stronach:

Awesome Oscillator (ang.)

Elliott Wave Oscillator (ang.)

Zastosowanie Awesome Oscillator w ramach wskaźnika Accelerator/Decelerator (pol.)

Jako że jest to pierwsza wersja implementacji, błędy są jak najbardziej możliwe. Prosiłbym o kontakt w przypadku ich wykrycia. Ze względu na ograniczenia ISPAG-a wskaźnik nie ma charakteru histogramu, ale oscylatora ciągłego. Poza aspektem graficznym nie wpływa to jednak na funkcjonalność.

Wersja wskaźnika w pliku tekstowym do pobrania dostępna jest w ramach repozytorium: http://blog.dvl.pl/snippets/ispag/

Reverse shell access

Problem

In my case this was a weird situation. I had a server I had to administer remotely [server C in the image] set up with some paranoid network security settings (no open remote SSH access). The only way to get to the sshd services was to use a special dedicated proxy [server B in the image] whose static IP address was excluded from the firewall list on the destination server (thus establishing a theoretically more secure SSH tunnel on some not so common port).

rrs.gif

The main problem was the latency resulting from such a construction. My route to the destination server [C] was only a few hops away and direct communication (ICMP) was always less than 20 ms. But the route via B was horrible, very unstable (some high-traffic node so latency spikes occurred every now and then). Normal SSH session got some 100-200 ms delay which meant usable, though not too comfy. But the spikes (2-3 seconds) utterly blighted any more complicated tasks being done on the server.

Solution

As I had a dynamic IP there was no easy way to grant me direct access to the machine in question. But I found a very interesting program, which enabled me to bypass the constraints established by the paranoid firewall setup.

Reverse Remote Shell by Michel Blomgren (Cycom AB) supports inverse SSH-like (OpenSSL) connections established between remote host [C] and the listener [my workstation - A]. As remote outgoing connections are fully accepted by the firewall on machine [C], I only had to set up a listener on my computer, connect through the old method (via tunnel) to the remote server and issue a reverse connection request to my current dynamic IP. When the connection got accepted, my listener transformed into a full-blown remote shell client and I could disconnect the tunnelled session enjoying near local latency of a direct connection.

Reverse Remote Shell is distributed under the MIT License and runs smoothly under Linux and BSD-family systems.

To make it as seamless as possible, I made a quick connection script for Linux.

First make a main connection script (something like ~/start.sh):

#!/bin/sh
# Establish a new screen session
killall screen
screen -dmS remoteShell
sleep 1
# Connect to the server via tunnel, when the session is completed - exit screen
screen -S remoteShell -p 0 -X stuff "~/connect.sh &amp;&amp; exit^M"
sleep 1
# This gets executed on the remote server - request a connection to the listener
screen -S remoteShell -p 0 -X stuff "~/remote.sh &amp;&amp; exit^M"
# Turn on listening on local side
~/listen.sh

You will also need a tunnel connection script (connect.sh). This can be something so trivial as:

#!/bin/sh
/usr/bin/ssh -p 22 -i ~/.ssh/myIdentityFile user@example.com

The remote part (remote.sh) is something like this:

#!/bin/sh
/bin/sleep 1
# Check your rrs location here, 12345 is a port of the listener
# and dynamic.example.com is a dynamic hostname for the listener (client)
~/rrs --daemon --ssl --pem ~/rrs.pem --timeout 15 dynamic.example.com 12345

And the last part – listener (listen.sh):

#!/bin/sh
~/rrs --listen --ssl --pem ~/rrs.pem --port 12345

This way a lot of paranoid firewall setups can be penetrated. Be aware, however, that this would probably need some reworking for corporate scenarios (even more strict as they are) and could potentially lead to some serious security level degradation. Notwithstanding, Reverse Remote Shell can be a really useful tool in such situations, so big thanks goes to Michel Blomgren for that kind of unique application.

Download

Author’s website: http://www.cycom.se/dl/rrs

Mirrored file: http://blog.dvl.pl/files/rrs-1.70.tar.gz

MD5: b400d03c0e39e3e78a7327ba78f789f0