Importing External logs into Rivendell - Music and Traffic

Currently alot of this page is a collection of emails and answers from the list and needs work to make it into a proper wiki page.

Which Schedulers Work with Rivendell

Q: I have a station that is running a Satellite music format but I would like to start scheduling my own music. The scheduler that I am familiar with is Selector. I would like to know if anyone is using an external music scheduler with Rivendell . if so I have some questions.

A1: We use a program called Music1. Rivendell imports it beautifully as log output file is totally configurable.

A2:I can't speak for it personally, but there is another one called Natural Music. They also make traffic software that I know interfaces with RD, so I would assume the music does as well. It's reasonably priced and can even be used for unlimited stations for not too much more than a single station license. It's a buy-out and not a leased program.

A3: Donna Halper & Associates in Boston has a scheduler called Results 98. It is a buyout program - no lease charges - and Halper is a well-respected broadcaster, author and historian.

A4: Just to chime in on this I have used Natural Music before we switched to Music1 and it worked well with Rivendell RDLogManager, We still use Natural Log for Traffic and it not only exports a file that RDLogManager can read but will also import the Technical Playout Report for reconciliation so your billing is correct.

Setting up Natural Music

Can any one provide/add samples of Natural Music logs and a screen shot of this setup in RDAdmin>Manage Services> select the service >Edit ?

Q: I am wanting to setup Natural Log and Natural Music. First thing I need to know for Natural Music is the location of the text file with the music's data. I can't find it.

A1: See the very good guide in the Natural Log™ 8 for Windows User Manual, Chapter 10 - Automation Interfacing, Section 10.41. It may help to look at the screen shot of Edit Services below along with the Edit Services part of the directions. You will also want to read the section below called Scheduling the external log parts in events in RDLogManager to learn how to get the external events into your log.

A2: In Natural Music go to setup and select station and in that window is a field for Automation Log send path, this is where the music log output is placed. You can set this path anywhere you want it to go. local disk or network path.

One of the things you can set in both programs is automation type for the automation output file. For Rivendell just set this to ascii text.

It is important that in Natural log you set the automation log send path to the same directory as the music because when Natural Log sees the Music log there it will merge the two into a combined music and traffic log.

Usually the traffic log file will end with "t1" and the music log file will end with "m1" The combo log will end with "c1" ie: 021508t1.txt 021508m1.txt 021598c1.txt if you have two stations natural log and music will end these with "t2" "m2" and "c2"

Once you have the final combo log open the text file and exam it to find the column that each data item you need is in. ie: cart number, Start time in hours minutes seconds, title etc. Count the column and length of each of these and write it down so you can put this info in Rivendell.

The setting for Rivendell are shown below. Enter this information in RDAdmin>Manage Services> select the service >Edit choose either Music Data Import or Traffic Data Import section. They are the same functionally. The information above was left to help non-Natural Log users figure out there settings.

Natural Music Data Import Settings for Edit Service

Field	Offset	Length
Cart Number	9	8
Title	19	24
Start Time-Hours	0	2
Start Time-Minutes	3	2
Start Time-Seconds	6	2
Length - Hours	62	2
Length - Minutes	64	2
Length - Seconds	67	2
All Other Fields	0	0

Open RDAdmin in Rivendell and select manage services, if you have not set up any others (and you don't have to) the default service is production. Select it and select edit. On this window you will see a section for Traffic data import and a section for Music data import. These are identical and can be use to import from any external system no matter what you are scheduling for. You will only need one of these if you are going to import a combo log, I suggest use music.

You can do them separately but it is really less hassle to just use the "c1" or combo log.

On each needed field put in the number you got from the text file above. offset is the column and length is the length of the field.

Put the network or computer path to the text file (log file) in the "linux import path" box.

You can now use the Test Music button to see the results, adjust column and length as necessary.

After you have run a log you can reconcile it back to Natural traffic. Follow instructions for reconciliation in Natural Log. Be sure to set the automation reconciliation type to Rivendell it is listed in the latest version of Natural Log. In order to use this you have to set up reports in Rivendell, instruction are in the manual. Natural Log reads the Technical Playout Report from Rivendell.

Log Import Format Presets

Starting with Rivendell 1.7.0 Preset import configurations have been added in RDAdmin->ManageServices for several third-party scheduler systems. Just select the preset and assure that your external traffic system is putting out the expected format. The following systems are supported. Click on the system names below to go to the wiki page describing the input and output formats for these.

Frequent Setup Issues

Situation: I want to use rdlogmanager to import music data from a text file in the following format:

hh:mm:sscccccc 

... where cccccc is the cart number and hh:mm:ss the time I want it to run.

In rdadmin under Edit Service, I've made the following settings:

Log Template name: %y%m%d 

Cart Number:          Offset: 9 Length: 6 
Start Time - Hours:   Offset: 1 Length: 2 
Start Time - Minutes: Offset: 4 Length: 2 
Start Time - Seconds  Offset: 7 Length: 2 

Music Data Import: Linux Import Path: /home/scott/music

The text file I want to import is called 090203, and it's in /home/scott/music and readable. However, rdlogmanager isn't finding it. The little indicator shows green after I click on Merge Music, but none of the carts shows up. I suspect my problem is that the file name needs some sort of suffix, but I can't guess what it might be. Does anyone know?

Offset Values Start at 0 NOT 1

A: Your offset values are all off by one. The values start at '0' not '1', so you'd want:

Cart Number:          Offset: 8 Length: 6 
Start Time - Hours:   Offset: 0 Length: 2 
Start Time - Minutes: Offset: 3 Length: 2 
Start Time - Seconds: Offset: 6 Length: 2 

A trick I use when setting up a configuration for a new file layout is to open the file in an editor that has a column counter (such as GNU Emacs). Then, just place the cursor on the first character of each field, and the column counter will give you the correct offset value to use. Saves a lot of tedious (not to mention error-prone) counting. On Ubuntu try Scribes Text Editor.

You Must Have a Filename Wildcard

Response: Alas, even with the offsets corrected it still won't import anything. I wonder what I'm doing wrong.

A: That is because you have not told Rivendell what for a filename, of the log, to look to import. Only what folder it is in.

Log Template name: %y%m%d only tells Rivendell to rename any newly created log in the MySQL database to %y%m%d.

But the name of music log you want to import is set in the Linux Import Path or the Windows Import Path (there is a Windows import path because we have RDLogManager for Windows) under either the Music Data Import or Traffic Data Import sections.

For your data path:

Music Data Import: Linux Import Path: /home/scott/music 

It should be something like:/home/scott/music/%y%m%d.log

%y = 2 digit year
%m = 2 digit month
%d = 2 digit day

TIP: File names must be absolute and not relative.

Linux Example: Your log files are located in /home/rduser/externallogs Then you can NOT enter externallogs as the path part of the log file template name as it is relative to where the command is run from. You must use the full path, starting at the root of the file system, of /home/rduser/externallogs it is absolute.

Windows Example: Your log files are located in C:\Users\rduser\externallogs Then you can NOT enter externallogs as the path part of the log file template name as it is relative. You must use the full path, starting at the root of the file system, of C:\Users\rduser\externallogs it is absolute.

Edit Services Screen Shot

Here is a screen shot of the Edit Services dialog from RDAdmin>Manage Services> select the service >Edit

Scheduling the external log parts in events in RDLogManager

How do items get into a log in RDLogManager? It is a building process even for imported items.

1. Create Events for your Clocks
2. Create Clocks for your Grids
3. Add Events to your Clocks
4. Add Clocks to your Grids
5. From Grids you produce Logs

More details on this process can be found on the wiki page Log Creation (adding static events) or on this Music Selection for Log (Adding events that are selected, like music).

Creating an Event for imported items

Below is a screen shot of an Edit Event dialog. Look carefully at the Import section.

Possible choices are:

1. None
2. From Traffic
3. From Music
4. Select From

In the screen shot a static event, a station ID, is being scheduled and None is selected. It will insert this station ID cart into the log where ever it is placed on a clock.

For imported schedules we will use From Traffic and From Music. These are directly related to the Traffic Data Import and Music Data Import sections in the Edit Services dialog, that we see a screen shot of above. So, if you set you external import up in the Traffic Data Import, you will want to tick From Traffic in the Import section when you create an event to import it. For Music Data Import tick From Music.

Then set the parameters for the Import in the section below that un-grays when you tick either the From Music or From Traffic box.

Need more details here. Can anyone help out and provided more info on these settings?

For example if you have 4 blocks of commercials that you schedule each hour, one at 0:15, 0:30, 0:45, and 0:00, You would need to insert this event into your clock at these four times to "look" for your events and import them into the log. I believe with importing external events you can import multiple external events into one event. This is not true of internally selected events like music where you can only select one cart for each event using the selection criteria.

Continue on with the information on these two pages, Log Creation (adding static events) or blog (Adding events that are selected, like music) to set up your Clocks and Grids so you can build your logs.

Another Explanation of How to Make This Work (Can you ever have enough?)

Rivendell is a very customizable application designed to work with a variety of different situations. Different users do different things.

While basically the same, all schedulers work slightly different. Further, different users may use the same scheduler in different ways. We can only speak from experiences we are familiar with. To me, a music scheduler provides a playlist. There are many things in a broadcast day besides music, depending on your station and its format. You may have IDs, jingles, features, spots, community calendars, and a whole bunch of other things to deal with. In many cases, these are not included in the schedule provided by the music scheduler. So, you put these items in your event list and leave "holes" to be filled by your music scheduler. Likewise, you may have "holes" to be filled by your traffic scheduler.

Now, if you have a scheduler that takes care of literally everything, including IDs, features, spots, PSAs, etc. Then you simply have one big hole to fill for the entire hour. You can make one event, put no carts in it and check the "import from music" box. Then you put that event in a clock and tell it to fill the whole hour. Next, put that clock into every hour in your grid, and you're done. Put your import data in the appropriate folder indicated in the services set up. Also in the services setup, you put in the necessary wildcards for the date information for each log and set up your "chainto" box.

Then all you have to do is open RDLogmanager, click "Generate Logs", merge your music import and you're done. It is still more than one step, and it isn't going to happen automatically. If you were good at programing, you could probably come up with a script to set up a "drop box" for your logs which would watch the import folder and automatically generate a log whenever an import file was dropped into that folder, but that presently is not a built-in feature of Rivendell. I suppose you could submit a feature request into Mantis and ask for it.

Also, there is not a function to automatically reconcile a scheduler's database with the Rivendell database. Due to the variety in configurations, this would have to be set up on a case by case basis, and there is no guaranty that it can be done in every instance. I think that might be beyond the "community support" aspect of OSS. You may have to hire a software contractor to come up with such an interface. Otherwise, you'll simply have to insure people input new carts into both databases.

In RDAdmin > Manage Services, your log template name should look something like "SR1_%y%m%d". This will generate a log named SR1_20070319. Similarly, your import path would look like "/home/salem/traffic_import/%y%m%dSR1.txt". Your scheduler would create a text file named 20070319SR1.txt and put it into the /home/salem/traffic_import/ folder. If you are using the Windows interface for Rivendell, you would have the appropriate Windows path and the folders would be mapped between machines as needed. Naturally, your actual filenames and paths will differ, and you would be using the music import instead of traffic import, but the functionality is basically the same.

Your description of your present problem sounds to me like you may not have set up your date variables properly. If your import file is correct, you may get a good test in RDAdmin>Manage Services, however, if your Log Template Name is not set correctly, you may not get a proper merge.