Encoding Videos For Zaurus, Improved Bash Script Options

[vrejakti]

nice code Da_Blitz, very nice! Admittedly, I am still learning how to write bash code, and the difference in professionalism between our scripts is quite crushing to me, but I don't mind. ^^ I'll keep improving my script as I learn more.

Keep up the good work.

[Da_Blitz]

thanks for that

when i switched to linux i did it the hard way and tried to install debian as a noob the reason was that i had to do a course on scripting for my egineering course. scince then i havent looked back and prefer to work with the command line

probelly the best intro to scripting is the read the advanced bash scripting guide (where i got the command line arg passing from) and the alien guide as well as the man pages for bash

links:
http://subsignal.org/doc/AliensBashTutorial.html
http://www.tldp.org/LDP/abs/html/

read i bit at a time but most of it comes from having a problem to solve, such as video encoding. the advanced bash scripting guide is great as it shows you best practicies and how to make your code portable (between shells and diffrent OS's)

for a bit of color (providing the $TERM enviroment setting is correct) try the tput command. here is a bit from my zsh startup scripts (i manage it in a simmilar way to init)

CODE

       GREY_TXT=`tput setf 7`
       RED_TXT=`tput setf 1`
       GREEN_TXT=`tput setf 2`
       YELLOW_TXT=`tput setf 3`
       BLUE_TXT=`tput setf 4`
       PINK_TXT=`tput setf 5`
       CYAN_TXT=`tput setf 6`
       WHITE_TXT=`tput setf 7`
       NORMAL_TXT=`tput setf 8`

       BOLD=`tput bold 1`
       BOLD_OFF=`tput bold 0`
       DIM=`tput dim 1`
       DIM_OFF=`tput dim 0`
       UNDERLINE=`tput smul`
       UNDERLINE_OFF=`tput rmul`
       REVERSE=`tput rev 1`
       REVERSE_OFF=`tput rev 0`
       ALTERNATE=`tput smacs`
       ALTERNATE_OFF=`tput rmacs`

and my prompt:

CODE

PS1="$BOLD$WHITE_TXT,-$BLUE_TXT [$RED_TXT%* %D$BLUE_TXT]-[$GREEN_TXT%n@$GREEN_TXT%M$BLUE_TXT]-[$PINK_TXT%y$BLUE_TXT]-[$RED_TXT%d$BLUE_TXT]$WHITE_TXT-,
'-$BLUE_TXT [$RED_TXT%?$BLUE_TXT] $RED_TXT%# $WHITE_TXT>- $WHITE_TXT"
RPS1=""

CODE

,- [19:24:23 06-09-14]-[dablitz@dhcppc7]-[pts/2]-[/home/dablitz/.zsh/startup]-,
'- [0] % >- cat 90prompt

your PS1 string is a good way to learn scripting and the builtin text manipulation abilities of the shell you use, keep in mind however that a 2 line prompt has a problem or two and i haveyet to see a shell that can handel without problems, i just live with it

[vrejakti]

Phew, looks like I don't have to implement ogg or mkv support into the script after all.

From the legendary Gentoo Wiki: http://gentoo-wiki.com/HOWTO_Mencoder_Introduction_Guide

"Ogg or Ogg bitstream format is also an open-source video container, part of the Xiph project. OGM is an extension of Ogg bitstream format to support some proprietary video codecs. Like matroska, mplayer can play, but not create Ogg and OGM videos."

"Matroska is an open-source video container, similar to AVI, except that it has many more advanced options and settings that can be included in the metadata. mplayer and mencoder can play and read matroska files, but not create them. Matroska audio and video files use filename extensions .mka and .mkv, respectively."

So, as of yet, mencoder and mplayer can read and play mkv or ogg file formates, but they can not create them.

Da_Blitz thank you for pointing out Aliens Bash Tutorial - that's the perfect resource to bring me up to speed on bash scripting. I've played around with the ABS guide for about a year now, and it's great, but Aliens guide looks to be the perfect compliment to it.

As for your shell prompt, that has got to be the most customized shell prompt I have ever seen.

[Da_Blitz]

thanks as i said it causes problems but as its hard to tell which machine i am on it cames in handy

i am glad i found ABS, it really helped to cement the stuff i learnt from the bash guide, if you get a chance readup on sed from an oriley book and perhaps awk, however if you think awk is a good idea then perl or python are probelly a better idea

with rivision 2 of the code i will be able to support multiple encoding egines (mencoder or the mkv or ogg stuff) so we will see if we need support for it <manical laugh>

i must say that the gentoo site is great, i dont think many people give gentoo enough credit as it has some of the best infomation of any distro i have encountered

[Da_Blitz]

Here is version 2.0, fixed a problem with the command line parsing and added code to actually encode the file

whats left is post processing or additinol options and cropping/scaling support as well as automatic sellection of the correct encoder for the specified format, however i suspect it will be implmented with function calls to the encoder profile and require a bit of a rewrite of the stuff that has been added

apart from that have fun and please poke holes in it, i expect that some areas can be slightly improved and i suspect there is a bug or two in there. in its current state it should reencode a moive if you set up the policy correctly.

CODE

# Copyright: Jay Coles 2006
# Licsence: GPL v2
# Version 0.9

# ######## Cut and paste from here into config file ##########
# ***** File Vars *******
ENCODER=mencoder
# File to be converted
INFILE=""
# Save file as
OUTFILE=""


# ****** Encoding Vars ******
# nice easy policys for encoding diffrent media
# added because it was easy to hack together :)
# note that you might want to add a policy_ to the
# front of your funtions to avoid clobbering
# the internal code
POLICY=policy_anime
POLICY_LIST="$POLICY policy_anime "

policy_anime() {
       NAME="Anime Medium Quality"
       DESCRIPTION="This is designed for encoding media for a zaurus with an anime file with hard subs (hence the high res)"
       AUTHOR="Jay Coles"
       LICSENSE="GPL"

       CONTAINER="avi"
       V_CODEC="xvid"
       A_CODEC="mp3"

       V_BITRATE="300"
       A_BITRATE="96"

       #Diffrent encoding policies (ie cbc abr vbr multipass)
       V_POLICY="multipass"
       A_POLICY="cbr"

       # allow recording at diffrent rates/fps
       # this really helps with anime by decimating the fps
       # by 2, allows for huge reductions in size with a small
       # loss in percived quality (ie motion is slightly more
       # jagged)
       V_RATE="0.5"
       A_RATE="22050"

       # Allow overiding of ratio blank for keep original
       RATIO=""
       KEEP_RATIO="y"
       # Note that these are max vals that will never be exceeded
       # while still maintiaining asspect ratio
       X_RES="640"
       Y_RES="480"

       #Amount of degrees to rotate video (helps increse performance)
       ROTATION=90
}

ENCODER_POLICY=encoder_mencoder

encoder_mencoder() {
       ENCODER_PROG=mencoder

       PROVIDES="xvid divx mp4"

       V_CODEC_STRING="-voc"
       A_CODEC_STRING="-aoc"
}
################ End cut and paste ################


################ functions ################
locate_codec() {
       for I in $ENCODER_POLICY; do
               . $I
               if (echo $PROVIDES | grep $1) = 0; then
                       echo $I
                       exit 0
               fi
       done

       exit 1
}


if [ -f /etc/zencoder.conf ]; then
       . /etc/zencoder/zencoder.conf
fi

if [ -f ~/zencoder.conf ]; then
       . ~/zencoder/zencoder.conf
fi

help () {
cat << EOF
zencoder: encodes moives for small media players

Usage: zencoder [-h | --help] [-o <VAR NAVE>=<VAL>] [-a <audio codec>]
               [-v <video codec>] [-b <video bitrate] [-d <audio bitrate>]
               [-x <res>] [-y <res>] [-p <policy>] [-r <rotation>]
               INFILE [-f OUTFILE]

       -h --help       This menu
       -o              Allows you to overide built in varibles
                       Mainly for testing
       -a <audio codec>
                       The audio codec used to encode the audio
                       use help for an abreaveated listing
       -v <video codec>
                       The video cadec used to encode the video
                       use help for an abreaveated listing
       -b <video bitrate>
                       The bitrate used to encode the video
       -d <audio bitrate>
                       the bitrate used to encode the audio
       -x <res>        The max X resolution
       -y <res>        The max Y resolution
       -r <rotation>   The angle to rotate the video output
       -p <policy>     Which settings to use to encode the file
       INFILE          The file to be processed
       -f <OUTFILE>    The name to call the compleated file,
                       Note that if not specified then it is
                       Automatically detirmined from the video
                       codec
EOF

exit
}

if [ "$1" = "-h" -o "$1" = "--help" ]; then
       help
fi

for ARG in $@; do
       case $ARG in
               -o) $2; shift;;
               -a) A_CODEC=$2; shift;;
               -v) V_CODEC=$2; shift;;
               -b) V_BITRATE=$2; shift;;
               -d) A_BITRATE=$2; shift;;
               -x) X_RES=$2; shift;;
               -y) Y_RES=$2; shift;;
               -r) ROTATION=$2; shift;;
               -f) OUTFILE=$2; shift;;
               -p) POLICY=$2; shift;;
               *) FILES="$FILES $2"; shift;;
       esac
done

for I in $FILES; do
       test -f $I || ( echo "File: $I dosent exist, bailing out"; exit 1 )
done

# Auto sources Policy
$POLICY
$ENCODER_POLICY

# note that we pass FILES in a loop to the var INFILE and process in
# the loop locally as INFILE

for INFILE in $FILES; do
       $ENCODER $V_CODEC_STRING $V_CODEC $A_CODEC_STRING $A_CODEC $INFILE $OUTFILE
done

Attached File(s)

 zencoder.sh.tar.gz ( 1.86K ) Number of downloads: 70

 

[johnny_a]

The script above gives me this error:

CODE

./zencoder.sh: line 77: syntax error near unexpected token `='
./zencoder.sh: line 77: `               if (echo $PROVIDES | grep $1) = 0; then'

Any ideas? I use bash.

[vrejakti]

The script above  gives me this error:

CODE

./zencoder.sh: line 77: syntax error near unexpected token `='
./zencoder.sh: line 77: `               if (echo $PROVIDES | grep $1) = 0; then'

Any ideas? I use bash.

Huh, at first glance, I'm thinking you may have saved the config file into the shell script. I haven't give this much thought, so I'm probably completly wrong here: Half the code may be a template for a config file that you save your perferences in. The other half is the shell script that does the encoding, reading defaults from the configuration file.

[johnny_a]

Thanks!

I didn't bother to read through the script.. which I should have done, of course. thanks again.

[Da_Blitz]

i thoght that the ####### cut here ##### was obvious enogh

i am aware it dosent work 100% right now but i am very happy with the structure of the code, all that comes now is the meat of it

[vrejakti]

I've tweaked my script once again. Code is cleaner. As for features, I added
1) Option to quickly jump to encoding. Type x when asked to, and it'll skip the next 3 questions.
2) A bit of an easter egg that lets you rip the OP of an anime episode in very high quality. At the last menu when it asks "Start encoding?" type "op" then enter hh:mm:ss of the episode to rip to HQ. 00:02:00 would grab the first 2 minutes. I wrote this code when I wanted to show a friend the coolest OP's from the newest anime season.
3) Tried to make it clearer for errors and note the importance of simple file names.

Complex file names are a bit of an issue in bash. Make sure your infile is "simplefilename.avi" and not "[I AM] 4 c0m|>;e'x' fiL_e name~~~~!". Renaming it is eacy in a GUI, select the file, hit F2, and type in something sane. In the console, make sure you're using Bash or ZSH, then type a single quote, a couple unique first letter, then press tab to auto comple the name. Use mv to rename that file to something simple.

Posting all these scripts as text is getting pretty crazy. *laughs* Here's a direct link to a download.

Download:http://www.lan358.com/scripts/vrecoder.sh
View:http://www.lan358.com/scripts/vrecoder.txt

[Da_Blitz]

not bad at all, never did like interactive stuff in bash but it seems to work out alright (im a big fan of the use the flags and get out of my face philoshpy, espesially when doing a batch job like this)

i have been thinking about mine and might do a bit of an archtecture change so that it only encodes 1 moive at a time as its easy enaugh to write a do while or for i in on the command line in one line (or even an xargs) and it simplifies some of my handelling

perhaps in the futre we can set it up so that you use my script as the encoding back end and yours as the interactive?

[vrejakti]

Combining scripts is an interesting idea. For now, I'll take a good hard look at your code and see what ideas I can come up with. Your script has a lot of features I like, such as the custimization options and the flags. At the moment, I'm looking for what code would be needed so a user simply types "./zencoder movie.avi" for the first time and the script takes care of the rest.

It should be possible to have the script make the dir $HOME/zencoder and use the same cat << EOF trick to drop the config text into $HOME/zencoder/zencoder.conf

All this remeinds me, I wanted to ask you what you would think of a ncurses interface for this whole script? Could be overkill, but it could also be useful in the sence of having a menu to check off which codec to use and such?

[vrejakti]

Several tweaks later... I've got your script encoding a file after typing ./zencoder -i video.avi -o z-video.avi The tweaks are barely proof of concept, but you may find something worth building on.

http://lan358.com/scripts/zencoder.txt

[vrejakti]

In a couple days I'll be releasing a new version of vrecoder that has two new features:

First off, it can handle any file name. So if you happen to have a lot of files named similar to [S^M] Galaxy Ange-lune 04 RAW.avi, the new script will have no issues at all with them.

Second, the script will be able to determine what files in the current directory are video files, and it will encode everyone of them. Obviously, this feature will be very useful if you want to convert your entire collection of videos over night.

Anyone who'd like a head start is welcome to hack the following code into a script of their own. Also, the shell zsh will be required to run the new script, so please grab it from
http://www.zsh.org/
or by using the package manager of your distro.

CODE

#!/bin/zsh
SMART_CODE=smart_code
smart_code () {
i=1
while read F
do
T=$(file -zibkp "$F")
minutes="00:01:00"
[[ $T == 'video/x-msvideo' ]] && mencoder $F -ovc lavc -lavcopts debug=0:vcodec=mpeg4:vbitrate=6000 -oac copy -endpos $minutes -o Zaurus-$i.avi
i=$[$i+1]
done < <(find -maxdepth 1 -printf "%f\n")
}
$SMART_CODE

The above code is a proof of concept for the two features. ;-)

[Da_Blitz]

any particular reason for zsh? i try hard to keep my dependencies down to bash

anyway back on topic, i did a rehash of my script and removed the ability to encode more than one file at a time.

other changes include a config file redesign so that the config file consists of a couple of varibles and one function (encode())

the reason for this is that several of the encoders handel things very diffrently so the codec choice has been moved to the encoder (ie no more chosing mencoder as the backend and allowing the "anime" policy to chose the codec

the policy now only choses bitrate, this was because of the incompatability of some mixtures and working out the match up was a PITA. in futre builds i might just merge this as well so that you only have to trade one file. actually that sounds like a better idea i might do that

funny thing is that mplayer already has support for decoding/encoding profiles built in so all my script is goot for is non mencoder support and automating multipass.

you think i should just release optimised mencoder profiles to go into the .mplayer folder i dont think mkv is a biggy at the moment (althogh i do prefer it) and ogg support would be nice however getting this stuff to work with streams is a nighmare (ie demux each steram from the container, rencoding the streams then reassemble into new container format), my current code will do it nicly but is it worth the effort (ie how many people want ogg and mkv support over avi and aac/mp3 with mpeg4)