mini REST HTTP server for launch scripts

Ideas and dreaming will go this forum

manos78
Posts: 21
Joined: Sun Jul 26, 2015 8:38 pm

mini REST HTTP server for launch scripts

Post by manos78 »

Hi,

I like to post here a mini REST/HTTP server based on NetCat for execute ANY arbitrary script in your environment.

I created this for call the good libAlert using simple HTTP requests (using CURL from any other device). This is best than use TELNET or SSH, as it don't needs any other external command in the remote device. Also is fast. Obviously, it has limitations: no secure, and BLOCKING (only one client at time). However it works robust!

You only need to condifure the script, copy in your environment, and start the server from your etc/init.d directory (I suggest to create a custom mini-rest-server.init). Perhaps you like to incorporate this in future versions of SamyGO!

Here the script...
mini-rest-server.sh:
SpoilerShow

Code: Select all

#!/mnt/bin/bash

#
# Simple REST/HTTP Command server based on NetCat (non multithreaded!)
# (c) 2015,Manos78 @ XDA
# Based on: http://stackoverflow.com/questions/26455434/create-a-minimum-rest-web-server-with-netcat-nc
#

# Configure for your environment!
TMP_DIR=/tmp
SERVER_PORT=5500
SODIR=/mnt/opt/privateer/usr/so


# Internal vars
FILENAME=${0##*/}
BASENAME=${FILENAME%%.*}
PID_FILE=$TMP_DIR/$BASENAME.pid
OUT_FILE=$TMP_DIR/$BASENAME.out
NCP_FILE=$TMP_DIR/$BASENAME.ncp

#echo $PID_FILE
#echo $OUT_FILE
#echo $NCP_FILE


# Controlled exit
trap "rm -f $PID_FILE; rm -f $OUT_FILE; rm -rf $NCP_FILE; exit" EXIT SIGINT


# Start of script
rm -f $PID_FILE
rm -f $OUT_FILE
rm -f $NCP_FILE
mkfifo $OUT_FILE || exit
touch $NCP_FILE || exit

pidfile() { (
    echo $BASHPID > "$1"
    shift
    exec "$@"
) }

echo "* Starting with pid: $$"
echo $$ > $PID_FILE

while true
do
  echo "executing netcat service: listening (oo sec.) for incoming requests"
  # On plain NetCat inside Busybox use simple parameters
  #cat "$OUT_FILE" | pidfile $NCP_FILE nc -vvvv -n -l -p $SERVER_PORT > >( # parse the netcat output, to build the answer redirected to the pipe "$OUT_FILE".
  cat "$OUT_FILE" | pidfile $NCP_FILE nc -l -p $SERVER_PORT > >( # parse the netcat output, to build the answer redirected to the pipe "$OUT_FILE".
    export REQUEST=
    while read line
    do
      line=$(echo "$line" | tr -d '[\r\n]')
      if echo "$line" | grep -qE '^GET /' # if line starts with "GET /"
      then
        REQUEST=$(echo "$line" | cut -d ' ' -f2) # extract the request
      elif [ "x$line" = x ] # empty line / end of request
      then
        HTTP_200="HTTP/1.1 200 OK"
        HTTP_404="HTTP/1.1 404 Not Found"
        HTTP_404="HTTP/1.1 400 Bad Request"
        HTTP_LOCATION="Location:"
        # call a script here
        # Note: REQUEST is exported, so the script can parse it (to answer 200/403/404 status code + content)

        # Testing echo service
        if echo "$REQUEST" | grep -qE '^/service/test/echo/'
        then
            printf "%s\n%s %s\n\n%s\n" "$HTTP_200" "$HTTP_LOCATION" "$REQUEST" ${REQUEST#"/echo/"} > "$OUT_FILE"

        # Testing data service
        elif echo "$REQUEST" | grep -qE '^/service/test/date'
        then
            date > "$OUT_FILE"

        # Testing stats service
        elif echo "$REQUEST" | grep -qE '^/service/test/stats'
        then
            vmstat -S M > "$OUT_FILE"

        # Your "message" service (using libAlert)
        elif echo "$REQUEST" | grep -qE '^/service/message/'
        then
            IFS='/'
            array=( $REQUEST )
            type=${array[3]^^}
            text=${array[4]}
            blink=${array[5]^^}
            blinks=${array[6]}
            delay=${array[7]^^}
            delays=${array[8]}
            if ! echo $type | grep -qEx '^TEXT|MSGBOX|SYSTEM|CENTER' ;
            then
               printf "%s\n%s %s\n\n%s\n" "$HTTP_400" "$HTTP_LOCATION" "$REQUEST" "REST API: Bad Request!" > "$OUT_FILE"
            else
               if ! echo $blink | grep -qEx '^BLINK' ;
               then
                  blink=""
                  blinks=""
               fi
               if ! echo $delay | grep -qEx '^DELAY' ;
               then
                  delay=""
                  delays=""
               fi
               echo "call $SODIR/libAlert $type:\"$text\" $blink:$blinks $delay:$delays"
               "$SODIR"/libAlert.sh $type:\"$text\" $blink:$blinks $delay:$delays
               printf "%s\n%s %s\n\n%s\n" "$HTTP_200" "$HTTP_LOCATION" "$REQUEST" "REST API: OK" > "$OUT_FILE"
            fi

        # Unkown request!
        else
            printf "%s\n%s %s\n\n%s\n" "$HTTP_404" "$HTTP_LOCATION" "$REQUEST" "Resource $REQUEST NOT FOUND!" > "$OUT_FILE"
        fi

      # Restart and Close
      echo "closing connection... "
      kill -SIGINT $(cat "$NCP_FILE")
      fi
    done
  )
done
I hope you like it! ;)
Last edited by manos78 on Mon Jul 27, 2015 8:25 am, edited 2 times in total.
manos78
Posts: 21
Joined: Sun Jul 26, 2015 8:38 pm

Re: mini REST HTTP server for launch scripts

Post by manos78 »

Hi,

I developed this on my Debian PC. Now I see that my SamyGO don't have the BASH shell. :shock:
Please, someone can point to me a Busybox with BASH applet compiled on it?

Thank you!

PD: I suggest to SamyGO developers to include BASH in the Busybox by default. Why not include it from start? :(

PD2: You can download one BASH binary from http://magizian.freeshell.org/.droid/android-port-bash/
Then copy to your environment and modify the launch shell in the script for use the full path.

PD3: This BASH binary is for Android and has troubles. Please, someone can compile BASH for E series? :?:
Last edited by manos78 on Sun Jul 26, 2015 10:21 pm, edited 1 time in total.
sectroyer
Official SamyGO Developer
Posts: 6305
Joined: Wed May 04, 2011 5:10 pm

Re: mini REST HTTP server for launch scripts

Post by sectroyer »

SamyGO's busy box has "ash" :) It's "almost" the same ;)
I do NOT support "latest fw" at ALL. If you have one you should block updates on router and wait for it to STOP being "latest":)
If you want me to help you please paste FULL log(s) to "spoiler"/"code" bbcodes or provide link(s) to pasted file(s) on https://pastebin.com Otherwise "NO HELP"!!!
If you want root DISABLE internet access to your device!!!!
DO NOT EVER INSTALL FIRMWARE UPGRADE !!!!
manos78
Posts: 21
Joined: Sun Jul 26, 2015 8:38 pm

Re: mini REST HTTP server for launch scripts

Post by manos78 »

sectroyer wrote:SamyGO's busy box has "ash" :) It's "almost" the same ;)
I try it, but it don't work with this script, as this script needs support for "( ... )" i.e. SUBSHELL.

Please, compile BASH... alone or inside Busybox. I really need it for complete this!
sectroyer
Official SamyGO Developer
Posts: 6305
Joined: Wed May 04, 2011 5:10 pm

Re: mini REST HTTP server for launch scripts

Post by sectroyer »

manos78 wrote:
sectroyer wrote:SamyGO's busy box has "ash" :) It's "almost" the same ;)
I try it, but it don't work with this script, as this script needs support for "( ... )" i.e. SUBSHELL.

Please, compile BASH... alone or inside Busybox. I really need it for complete this!
ehh ? "subshell" ?:D
like this:

Code: Select all

$(ls /)
or this:

Code: Select all

`ls /`
?
Here work like a charm with SamyGO's busybox :)
I do NOT support "latest fw" at ALL. If you have one you should block updates on router and wait for it to STOP being "latest":)
If you want me to help you please paste FULL log(s) to "spoiler"/"code" bbcodes or provide link(s) to pasted file(s) on https://pastebin.com Otherwise "NO HELP"!!!
If you want root DISABLE internet access to your device!!!!
DO NOT EVER INSTALL FIRMWARE UPGRADE !!!!
manos78
Posts: 21
Joined: Sun Jul 26, 2015 8:38 pm

Re: mini REST HTTP server for launch scripts

Post by manos78 »

sectroyer wrote:
manos78 wrote:
sectroyer wrote:SamyGO's busy box has "ash" :) It's "almost" the same ;)
I try it, but it don't work with this script, as this script needs support for "( ... )" i.e. SUBSHELL.
ehh ? "subshell" ?:D
like this:

Code: Select all

$(ls /)
or this:

Code: Select all

`ls /`
?
Here work like a charm with SamyGO's busybox :)
Yes! The problem is the different ( ... ) calls in the script: pidfile function, nc processing, the array processing of the $REQUEST, etc. Are you sure that in ash `...` works equal? Have you tested my script? Please, can you help me to port to ASH?

Please, help me to improve this... I feel will be very useful to include more remote control of the TV! :idea:
manos78
Posts: 21
Joined: Sun Jul 26, 2015 8:38 pm

Re: mini REST HTTP server for launch scripts

Post by manos78 »

Hi sectroyer,

I'm traying to translate from BASH to ASH inside my Debian PC, but it's very frustrating! The "Subprocess Substitution" ins't present in the ASH shell, and this script is based on this.

Please, can you help me to complete the port?

Thank you!
manos78
Posts: 21
Joined: Sun Jul 26, 2015 8:38 pm

Re: mini REST HTTP server for launch scripts

Post by manos78 »

Hi,

Here my translated version to ASH, not finished at time!!!

mini-rest-server-ash.sh:
SpoilerShow

Code: Select all

#!/bin/ash

#
# Simple REST/HTTP Command server based on NetCat (non multithreaded!)
# (c) 2015,Manos78 @ XDA
# Based on: http://stackoverflow.com/questions/26455434/create-a-minimum-rest-web-server-with-netcat-nc
#

# Configure for your environment!
TMP_DIR=/tmp
SERVER_PORT=5500
SODIR=/mnt/opt/privateer/usr/so


# Internal vars
FILENAME=${0##*/}
BASENAME=${FILENAME%%.*}
PID_FILE=$TMP_DIR/$BASENAME.pid
OUT_FILE=$TMP_DIR/$BASENAME.out
IN_FILE=$TMP_DIR/$BASENAME.in
NCP_FILE=$TMP_DIR/$BASENAME.nc.pid
CAP_FILE=$TMP_DIR/$BASENAME.cat.pid


# Controlled exit
trap "rm -f $PID_FILE; rm -f $OUT_FILE; rm -f $IN_FILE; rm -rf $NCP_FILE; rm -rf $CAP_FILE; exit" EXIT


# Start of script
rm -f $PID_FILE
rm -f $OUT_FILE
rm -f $IN_FILE
rm -f $NCP_FILE
rm -f $CAP_FILE
mkfifo $OUT_FILE || exit
mkfifo $IN_FILE || exit
touch $NCP_FILE || exit
touch $CAP_FILE || exit


split () {
    local str sep sub
    str=$1 sep=$2
    shift 2
    until test $# = 0; do
        sub=${str%%[$sep]*}; str=${str#$sub}; str=${str#[$sep]}
        eval $1='$sub'; shift
    done
}

echo "* Starting with pid: $$"
echo $$ > $PID_FILE

while true
do
  echo "executing netcat service: listening (oo sec.) for incoming requests"
  export REQUEST=
  cat "$OUT_FILE" | nc -l -p $SERVER_PORT | tee "$IN_FILE" &
  echo $!
  echo $! > $NCP_FILE
  while read line < "$IN_FILE";   # parse the netcat output, to build the answer redirected to the pipe "$OUT_FILE".
    do
      line=$(echo "$line" | tr -d '[\r\n]')
      if echo "$line" | grep -qE '^GET /' # if line starts with "GET /"
      then
        REQUEST=$(echo "$line" | cut -d ' ' -f2) # extract the request
      elif [ "x$line" = x ] # empty line / end of request
      then
        HTTP_200="HTTP/1.1 200 OK"
        HTTP_404="HTTP/1.1 404 Not Found"
        HTTP_404="HTTP/1.1 400 Bad Request"
        HTTP_LOCATION="Location:"
        # call a script here
        # Note: REQUEST is exported, so the script can parse it (to answer 200/403/404 status code + content)

        # Testing echo service
        if echo "$REQUEST" | grep -qE '^/service/test/echo/'
        then
            printf "%s\n%s %s\n\n%s\n" "$HTTP_200" "$HTTP_LOCATION" "$REQUEST" ${REQUEST#"/echo/"} > "$OUT_FILE"

        # Testing data service
        elif echo "$REQUEST" | grep -qE '^/service/test/date'
        then
            date > "$OUT_FILE"

        # Testing stats service
        elif echo "$REQUEST" | grep -qE '^/service/test/stats'
        then
            vmstat -S M > "$OUT_FILE"

        # Your "message" service (using libAlert)
        elif echo "$REQUEST" | grep -qE '^/service/message/'
        then
            split "$REQUEST" '/' first second third type text blink blinks delay delays
            type=$( echo $type | tr '[:lower:]' '[:upper:]' )
            blink=$( echo $blink | tr '[:lower:]' '[:upper:]' )
            delay=$( echo $delay | tr '[:lower:]' '[:upper:]' )
            echo $type,$text,$blink,$blinks,$delay,$delays
            if ! echo $type | grep -qEx '^TEXT|MSGBOX|SYSTEM|CENTER' ;
            then
               printf "%s\n%s %s\n\n%s\n" "$HTTP_400" "$HTTP_LOCATION" "$REQUEST" "REST API: Bad Request!" > "$OUT_FILE"
            else
               if ! echo $blink | grep -qEx '^BLINK' ;
               then
                  blink=""
                  blinks=""
               fi
               if ! echo $delay | grep -qEx '^DELAY' ;
               then
                  delay=""
                  delays=""
               fi
               echo "call $SODIR/libAlert $type:\"$text\" $blink:$blinks $delay:$delays"
               "$SODIR"/libAlert.sh $type:\"$text\" $blink:$blinks $delay:$delays
               printf "%s\n%s %s\n\n%s\n" "$HTTP_200" "$HTTP_LOCATION" "$REQUEST" "REST API: OK" > "$OUT_FILE"
            fi

        # Unkown request!
        else
            printf "%s\n%s %s\n\n%s\n" "$HTTP_404" "$HTTP_LOCATION" "$REQUEST" "Resource $REQUEST NOT FOUND!" > "$OUT_FILE"
            echo "ERROR"
        fi

        # Restart and Close
        echo "closing connection... "
        if [ -f "$NCP_FILE" ]
        then
            kill -2 $(cat "$NCP_FILE")
        else
            echo "Error, killing..."
            exit
        fi
      fi
    done
done
The problem to resolve is save (capture) the PID of the NetCat instance, and now also for the Cat command. The problem is access to PID values, because inside a pipe it's impossible to use the $! variable.

Please, if someone knows how to capture it, then try to fix the script. This is the only remaining function for support ASH shell!
sectroyer
Official SamyGO Developer
Posts: 6305
Joined: Wed May 04, 2011 5:10 pm

Re: mini REST HTTP server for launch scripts

Post by sectroyer »

Stop posing shit about "Subprocess Substitution" or full script :P Simply paste the line/part that doesn't work :P I code sh scripts for TV EVERY FUCKING DAY and I haven't found ANYTHING that doesn't work :P
I do NOT support "latest fw" at ALL. If you have one you should block updates on router and wait for it to STOP being "latest":)
If you want me to help you please paste FULL log(s) to "spoiler"/"code" bbcodes or provide link(s) to pasted file(s) on https://pastebin.com Otherwise "NO HELP"!!!
If you want root DISABLE internet access to your device!!!!
DO NOT EVER INSTALL FIRMWARE UPGRADE !!!!
manos78
Posts: 21
Joined: Sun Jul 26, 2015 8:38 pm

Re: mini REST HTTP server for launch scripts

Post by manos78 »

sectroyer wrote:Simply paste the line/part that doesn't work :P
Umm.... Sorry?

The "last" version, translated to ASH, it works! Please, try it!

The "problem" is that the script doesn't execute the correct behaviour. Why? Because at some point it needs to CLOSE the NetCat session. To execute this it needs to know the PID of the NetCat, but this is IMPOSSIBLE to obtain if you execute it on a PIPE. Also, with this version of the script translated to ASH the cat process that feeds the NetCat session needs to be killed at end.

Please, see this line of the code:

Code: Select all

cat "$OUT_FILE" | nc -l -p $SERVER_PORT | tee "$IN_FILE" &
This launches one "cat" process, one "netcat" process and one "tee" process. But then...

Code: Select all

        # Restart and Close
        echo "closing connection... "
        if [ -f "$NCP_FILE" ]
        then
            kill -2 $(cat "$NCP_FILE")
at this point of the code you need to "kill" the NetCat process (now also the cat process). Why? Because at end of the HTTP transaction (Request-Answer) the NetCat continues to maintain the TCP socket open.

And the remaining problem is: How to save in the $NCP_FILE the pid of the NetCat process?

If you know the solution, please, help me to fix this part of the code. This is the only remaining point for use this script! ;)
Thank you!

Post Reply

Return to “[E] Brainstorm”