Home : Website : Scripts : NZBGetPP :

Archive

This is the original NZBGetPP script. The script has been rewritten and reborn as the nzb and post processing portion of the PPWeb project.

Download

Requires NZBGet 0.4.0 (r149 or newer).

nzbgetpp.pl (03-Jun-2008)

See also: Change Log
[ page | comments (17) - Thursday, 03-Apr-2008 | top ]

$ScriptDir
      Where to put the scripts and logs.

$Rename
      The goal is to rename all suitable files using the nzb base name.
      Extracted files with a series or CD label (that nzbgetpp recognizes) get renamed.
      Downloaded files (not in rars) with extensions in SaveExt will be renamed to match.
      Nested rars (e.g. subs) will be extracted and renamed to match.
      See also: Automatic Renaming (and Flags).

$Delete
      Delete the download directory and nzb file.
      Done after extracting, renaming and moving/saving files to SaveDir
      Set to 0 until you know everything you want to keep is ending up in SaveDir

$SaveDir (was ExtractDir)
      Where to put extracted and saved files.
      See also: $MiscDir

$SaveSub
      When set to 1, each nzb will be extracted/saved to its own sub-directory.
      When set to 0, sub-directories will be created for mp3 collections only, you can
      use the " dir" flag to save/extract select collections to a sub-directory.

$SaveExt
      All extracted files are saved, this determines what other files get saved.
      A bar separated list of file extensions
      This is important because the script now handles all downloaded files

$MiscDir
      Send one or more saved extensions to an alternate directory

$MiscExt
      Which saved extensions to send to MiscDir

$DownLog
      A bar separated list of file extensions to log
      An empty list ($DownLog = '') disables it.
      The log is alphabetically sorted with each addition
      If $Rename = 1
            Renamed name gets padded to 55 characters
            A tab is added between the new and old file-name

$Avimerge
      When set to 1 and extraction results in two avi files that have CD labels,
      the files will be joined (unless there are subtitle files). Requires transcode.
      foo a.avi and foo b.avi > foo.avi (a and b get deleted if Delete = 1)

$Queue
      Allows for delayed and/or remote post processing.
      Instead of extracting/saving files, a shell script is created.
      You decide when and where to do the post processing.
      See also: Remote Processing

$Nice
      Unrar (and avimerge) takes a lot of system recources,
      nice -n19 tells the OS to make it a lower priority.

[ comment | link | top ]

NZBGet Configuration

In ~/.nzbget enter the path to the perl script:
      PostProcess=~/path/to/nzbgetpp.pl

The script assumes that NZBGet is taking care of par2 repair and cleanup:
      LoadPars=one
      ParCheck=yes
      ParRepair=yes
      ParCleanupQueue=yes

The script also assumes that:
      AppendNzbDir=yes
[ comment | link | top ]

Automatic Renaming

If $Rename=1, extracted and saved files are renamed using information nzbgetpp finds in the nzb and extracted/downloaded file names. While the surest bet is to 'save as'/rename your nzb using the show/movie title (e.g. Foo.nzb), nzbgetpp has gotten pretty good at finding the titles on its own (e.g. in nzbsrus nzb file names). With an nzb named Foo.nzb, extracted and saved files get renamed Foo.ext (Foo.avi, Foo.nfo, etc.).

The script also looks for episode and CD labels (S01E01, cd1, etc.) in the nzb and extracted/saved file names. If an episode/CD label is found, it will be attached to the title (Foo 101.ext, Foo a.ext). All episode labels get formatted as " (n)nnn" and all cd labels get formatted as " a" or " b". Because of the unique labels, multiple files with the same extension can be renamed (e.g. Foo.nzb > Foo a.avi and Foo b.avi).

Much of the code in this script is for the renaming functions. What you and I download could be quite different. There are likely to be naming schemes that the script cannot decipher. I need to know when renaming doesn't work. Please provide the nzb-name or an example file-name when something doesn't work as expected.

Example
      Nzb name:
            109254-National.Tresure.Book.of.Screts.DVDRip.XviD-DiAMOND-nzbsrus.com-2739.nzb
      Downloaded and extracted files:
            dmd-nt2-cd1.avi
            dmd-nt2-cd2.avi
      Auto renamed file names:
            National Tresure Book Of Screts a.avi
            National Tresure Book Of Screts b.avi
      Optional merged file (Avimerge=1):
            National Tresure Book Of Screts.avi

See Flags for mp3 renaming.
[ comment | link | top ]

Flags

Foo dir.nzb:
      Extract/move to SaveDir/Foo
      The script looks for " dir" before .nzb

Foo kbf.nzb
      For downloading the first few rar's of a collection
      The script looks for " kbf" before .nzb
      Extracts and saves as Foo sample.ext
      Removes the " kbf" from the download directory name so
      you can continue downloading later with a Foo.nzb (saved as Foo.ext)
      or you can manually delete the Foo directory
      ~/.nzbget should have DupeCheck=yes to prevent re-downloading of sample rars

Artist - Album mp3.nzb
      For renaming mp3 albums
      The script looks for " mp3" before .nzb and a single " - " separator
      Artist and Album need to be alphanumeric (...removed this restriction)
      Extracts and saves all (incl. cd sub dir's) mp3's to "Artist - Album" directory
      Song names are renamed/reformatted as n - Song Name.mp3
      CDn is prepended to song number, e.g. CD2 - 09 > 209
      Using just the " - " flag (Artist - Album.nzb) is just too risky at this point

Foo.nzb
      No flags needed ~~for single file extractions~~.
      If $Rename = 1, rename extracted and $SaveExt files to Foo.ext
      soMe-wacky.name.avi becomes Foo.avi (extracted)
      soMe-wacky.name.nfo becomes Foo.nfo ($SaveExt includes nfo)
      ...and now
      soMe-wacky.cd1.name.avi becomes Foo a.avi (extracted)
      soMe-wacky.S01E01.name.avi becomes Foo 101.avi (extracted)
      soMe-wacky.name.1x01.nfo becomes Foo 101.nfo ($SaveExt includes nfo)
      ...etc. The above 2 would also work if you named your nzb Foo 101.nzb
      ... or Foo.S01E01(.bar).nzb or Foo.1x01(.bar).nzb

Foo n-n.nzb... is now automatic, naming scheme is optional:
      Rename multiple collection episode/CD nzb's
      The script looks for " n-n" before .nzb
      Finds matches in file names, the numbers in the nzb-name don't matter
      Extract/move and rename to Foo nnn.ext if file-name contains:
            SnnEnn
            nnXnn
            nnn(n)
      Extract/move and rename to Foo a.ext, Foo b.ext
            foo.cdn(.bar).ext
            foo.n.ext, foo.a.ext, foo.b.ext
            foon.ext, fooa.ext, foob.ext (no separator, requires 2 matches before renaming kicks in)

Foo n-n dir.nzb... is now automatic, naming scheme is optional:
      Rename multiple collections to Foo nnn.ext or Foo a (or b).ext and place in SaveDir/Foo
[ comment | link | top ]

Remote Processing

Remote processing is setup so that processing can be done on another machine and at any time, i.e. the remote machine doesn't have to be running while nzbget is downloading.

If $Queue = 1, a shell script is written for each downloaded nzb (that's all it does). When you want to process the nzb(s), run the shell script(s). The shell script executes a remote copy of nzbgetpp.pl... I haven't tried it but I'd think you could also set it up for manually processing on the nzbget machine.

The script can be setup for testing this feature (if $Queue = 1) by changing line 80 from elsif to if. The script will write a shell script and do the usual post processing on the nzbget/local machine. To test the generated shell script make sure $Delete = 0. Since processing moves SaveExt files from the download directory, it won't be a 100% test unless you copy/rename those files back to the download directory before running the shell script... For more realistic testing you could use an empty $SaveExt (='') on the local machine.
[ comment | link | top ]

Wish List

Mp3 renaming - regular expressions that would allow mp3's to be renamed to a standard format (e.g. nnn - Foo Bar.mp3)... Initiated, unknown success rate... pretty good if artist and album names in nzb-name match any artist and album names in song titles.
Unzip - like nested rar's, extract everything.
Remote queue - One shell script instead of one for every nzb.
Directories - Extract and save to more than one directory (e.g. Video and Music).
Custom rename - Multiple choice renaming schemes.

[ comment | link | top ]

Change Log

03-Jun-2008: Changed configuration instructions. Not sure why I thought the shell script was needed, 'PostProcess=~/path/to/nzbgetpp.pl' in .nzbget works fine. Thanks to ydrol.

31-May-2008: While I've been using this for weeks with no trouble, there were a lot of changes and testing wasn't thorough.

Hack/fix for .nnn rar sets (numerical only extension). Those rar sets will require manual extraction unless they are more common than I think and a proper fix (auto extraction/rename) is merited.

Reversed download.log field order. Renamed name is now at the beginning of the line (improved sort). If converting existing logs to the new format is important, let me know. I haven't tested the new log format with rename off.

Added support for auto renaming of NZBsRus nzb's and did a number of rename tweaks.

Added more sampling (" kbf") options including auto-sampling. Add "kbf" and "hold" sub-directories to your "nzb" directory. If you want a sample, save your nzb to the "kbf" directory. If/when you want to download the rest, move the nzb from the "hold" to "nzb" directory.

No longer using nzbgetpp.log (Perl error messages were still being logged there). Slightly simplified configuration.

22-Mar-2008: Added $MiscExt check (ref).

21-Mar-2008:
Added CDn handing to " mp3" flag renaming, e.g. CD2 - 09 - song.mp3 > 209 - song.mp3.
Fix for no sub-directory when $SaveSub = 1 and $Rename = 0 (ref).

17-Mar-2008: Improved handling of mp3 renaming.

15-Mar-2008:
Improved rename handling of .ext dupes in nested rars (e.g. en.srt, fr.srt, etc.).
Improved handling for extracted dupes of downloaded (not extracted) files (e.g. nfos).
Fixed DownLog formatting of downloaded (not extracted) files.
Improved handling of failed extractions (KISS error message).
Added keep broken file - " kbf" - flag for downloading/extracting a sample.
Experimenting with mp3 renaming (see Flags).
Switched to using NZBGet log and web interface (or terminal) for all but system messages.
Requires r149 (svn) or higher.

11-Mar-2008:
Added special rename handling for the numbers 1940 - 2009 (much more likely to be years than episode labels).
Added $Nice to config.
Added a foonnn(n).(bar.)ext regex (some.name101.ext > nzb-name 101.ext).

Restored Foo n-n.nzb renaming. If using the " dir" flag (Foo n-n dir.nzb) or if SaveSub = 1, n-n will be included in the sub-directory name only, e.g. Foo 101-2( dir).nzb > Foo 101-2 sub-directory containing Foo 101.avi and Foo 102.avi. In all other cases n-n will be ignored.

07-Mar-2008: A number of tweaks and potential bug fixes. Tested rared and plain mp3 collections (w/ auto, " dir" and SaveSub)

06-Mar-2008 - 2: Changes to overwrite prevention including a proper fix for the special character bug.

Added a $SaveSub option. When set to 1, a separate save sub-directory will be created for every nzb. All extracted and saved files will be saved to $SaveDir/[nzb base-name] (foo.nzb = SaveDir/foo). This was added by request and has not been tested.

Added automatic sub-directory. When an nzb contains a collection of files with the same extension and no ep/cd label (e.g. mp3 albums), they will be saved to a sub-directory of SaveDir, i.e. the " dir" flag is optional in those cases (episode/CD collections still default to SaveDir).

06-Mar-2008: Contains a quick fix for bug caused by special characters in the nzb file-name.

05-Mar-2008: Committed Remote Processing. Added fooa.avi/foob.avi and foo1.avi/foo2.avi auto renaming. Avimerge is disabled if a .srt or .sub file is found.

04-Mar-2008: Changed Remote Processing.

03-Mar-2008: Adjusted split avi processing to work with auto rename. Added Remote Processing.

02-Mar-2008: Now renaming/moving SaveExt (was copy). Added $Avimerge. If you have transcode installed, cd1 and cd2 can be joined (Foo.nzb > Foo.avi).

01-Mar-2008: Duh, can't have \b in a character class, fixed a couple of regular expressions. Added a Foo.s01e01(.bar).nzb or Foo.1x01(.bar).nzb > Foo 101.ext regex.

29-Feb-2008 - 2: Getting there. Auto renaming using file ep/cd label tweaked and tested in depth for extracted and saved files, any failure will revert to original file-names. Renaming keeps extracted directory structure. Haven't retested split avi's or nested rars.

28-Feb-2008: Added auto renaming. While testing was minimal, its being released because it fixes a minor bug in yesterdays version and the code changes were relatively minor.

27-Feb-2008: Minor name / description changes (e.g. ExtractDir > SaveDir) to better describe enhanced functions. Added rename dupe checking.

26-Feb-2008: Lots of additions, read the rest of this page. I haven't been able to test all the angles so I would recommend $Delete=0 for now. To play it extra safe, the script is currently copying saved files (SaveExt) instead of moving them. Requires nzbget r129 (nzbget-testing) or higher.
[ comment | link | top ]

Browse