NEP technical guide

Massey University

Decisions an adverts

To accept an advert with a date yyyymmdd issue the command

ernad@aigtu:~$ cp ~/opt/chungju/orders/yyyymmdd.xml ~/var/chungju/orders/accepted/.

Use the tabulation character to see a list of available dates.

To refuse an advert with a date yyyymmdd issue the command

ernad@aigtu:~$ cp ~/opt/chungju/orders/yyyymmdd.xml ~/var/chungju/orders/refused/.

In principle, there should no need to inform anybody. The news about a decision should be up by the monitoring software within ten minutes.

Reset a report issue

Assume you want to completely reset the issue issue_date of report report

ernad@aigtu:~$ wipe_report_issue nepreport issue_date

This will also send a notification to the editor that a new version of th e presorted report is available. Therefore we need to consult with the editor first.

Static pages

After the Wellington reform of November 2012, master copies of the static page are held in the ~/www directory, as XML. Before making major changes to a page, make sure you copy the page’s old html file. On a day yyyymmdd, preserve the html of a page page by doing

ernad@aigtu:~$ cp ~/public_html/opt/page.html ~/public_html/old/page_yyyy-mm-dd.html

Static pages are generated with ~/ernad/perl/make_static_pages nep.

The main page is treated differently. Its code is mainly in the ~/xslt/service.xslt.xml.

List the papers queue

Since 2016‒08‒07, we run Yanabino protocol for issue smoothing. The queue is public.

Changing sponsor data

Sponsor data is still dispersed in three location.

Checking that the editors are doing their work

Thomas Krichel will, over the next few years, set up NEP_Watch, a comprehensive system that will allow to monitor what editors are doing, and who NEP as a whole performs. We already have

Checking email delivery

You can find the last time ernad called Mailman, and the last time exim attempted delivery of a message to subscribers. Normally, the former should be earlier than the later. Use last_mail_events report. If you leave out report, data for all reports is shown.

Create RSS file

When a report issue is mailed, the RSS file is generated automatically, when a new report issue is produced. It can be generated manually

ernad@aigtu:~$ make_rss report

At this time, the automatic generation does not work for nep-all. Therefore it has to be done manually.

Suspending an editor

You have to edit ~/ernad/etc/nep/reports.amf.xml and comment out the information about the editor. Change

   <ernad:password>...</ernad:password>
    <haseditor>                                                     

with

   <ernad:password>...</ernad:password>
<!--                                                                
    <haseditor>                                                     

and

   </haseditor>                                                    
  </collection>

with

  </haseditor>                                                    
-->
 </collection>

and restart ernad.

ernad@aigtu:~$ kill_ernad nep

To put the report back, put the information for the new editor, including the correct shortid in RAS, change

   <ernad:password>...</ernad:password>
<!--                                                                
    <haseditor>                                                     

with

   <ernad:password>...</ernad:password>
    <haseditor>                                                     

and

  </haseditor>                                                    
-->
 </collection>

with

   </haseditor>                                                    
  </collection>

then restart ernad

ernad@aigtu:~$ kill_ernad nep

Checking if a list works

A list “works” if it accepts posts from ernad@nep.repec.org are accepted. There are two ways to send a test mail.

First, you may want to send a simple string that you write out on the command line. In this case

ernad@aigtu:~$ echo "text" | mutt -s "subject" report@lists.repec.org

Here text is the text you want to send in the body of the mail, subject is the subject of the message, and report is a report. Example

ernad@aigtu:~$ echo "test mail 1" | mutt -s "test, please ignore" nep-ecm@lists.repec.org

Second, you may want to send the contents of a text file. In that case, compose the file in the ~/mail directory first, say in ~/mail/message or any other file called file. Then use

ernad@aigtu:~$ cat file | mutt -s "subject" report@lists.repec.org

Here file is the path to the file you want to send in the body of the mail, subject is the subject of the message, and report is a report. Example

ernad@aigtu:~$ cat ~/mail/statement_to_all | mutt -s "a word from the director of NEP" nep-all@lists.repec.org

Checking the mail queue

You can look at the mail queue with ~/perl/mailq. All lines with ' D ' represent mail addresses that have been delivered already.

Managing the publication of report histories

Until August 2006, the gathering of report histories was a very cumbersome process. There have been several kinds of log files parsed but all are mail files. They are therefore highly unreliable, for the pre-ernad period especially. All parsing was done by the script ~/perl/nepmailparse and the resulting AMF data stored in ~/public_html/amf/nep/, one file per report. These files are still there. They are used for the gathering of pre-ernad historical data, but they are no longer being updated.

The new historic file system is known as repsum. The main innovation that they introduce is that they credit individual editors for each issue. To aim is to get a complete history of all editors. This can be used to customize the experience length.

There are three scripts to maintain repsum data.

~/perl/repsum_mail parses the data from the email-based record files, in the old directory ~/public_html/amf/nep/. It adds historic editor information as it is maintained in the historic editor file ~/var/pre_ernad_editor_data.txt. ~/perl/repsum_mail only needs to be run when the date in the historic editor file is changed. How to do that is described in the next section. The ~/perl/repsum_mail generates some Perl output in ~/var/repsum/mail. This Perl output serves as input for repsum_all.

~/perl/repsum_ernad parses the sent directory for every report, ~/ernad/var/nep/reports/report/sent directory for every report report for historic records. To find the editors that are associated with report issues, it is sufficient to parse the report issue file (rif). If that information is not found in the rif, the script applies some backward heuristics. These heuristics relates most of this issues to identified (i.e. RAS-registered) editors. Those that are not resolved are reported upon by the script. The script generates some Perl output in ~/var/repsum/ernad. This Perl output serves as input for ~/perl/repsum_all.

~/perl/repsum_all parses the files in ~/var/repsum/ernad and ~/var/repsum/mail. It extracts the person’s short-id out of the homepage date, and splices the mail and ernad data together, adding mail data only when ernad data is not available. The script generates some AMF XML output in ~/public_html/repsum/. The output is self-explanatory, and mirrored to the conf directory of RePEc:wop.

It remains that one advantage of mail file parsing is that it gathers data when the recipients of the report have received it, rather than at the time when the report created. Thus current NEP historical records will contain data from reports that have never been distributed via email.

Managing pre-ernad editorial history

This simple text file should contains information about editors in the pre-ernad area, basically from April 1998 to January 2005. Lines that start with # are ignored. Other lines should have the form

report id start end name

where report is the report, id is the short-id of the editor, start is the start date, end is the end date, and name is the name of the editor. Here is an example

nep-ecm         pka1    1995-01-01      2006-01-01      Sune Karlsson

You can have more than one line per report, but please make sure that the editorial time periods don’t overlap.

When a change has been made to the file, a you have to run ~/perl/repsum_mail to make use of the changes in the data that is gathered. Note that changes to the file only affect the pre-ernad report history.

Undeleting a report issue

When an editor finds no relevant paper in a nep-all report issue, she can choose to delete the nep-all issue. When an editor pushes the delete button, the corresponding rif is not deleted. Instead, an empty rif is written into /home/ernad/var/reports/nep-xxx/sent. Here nep-xxx is the report id. Thus, if yyyy-mm-dd is the issue date, you just have to do

ernad@aigtu:~$ cd ~/ernad/var/nep/reports/nep-xxx

ernad@aigtu:~$~/ernad/var/nep/reports/nep-xxx rm sent/yyyy-mm-dd*

However, if the editor has also already gone through the creation of the issue, but wants to reissue the issue because, for example, there has been a problem with the distribution of the issue, all files for the particular report issue will have to be deleted. In that case see the next section.

Deleting a report issue

If an editor has gone through the creation of an issue, but wants to reissue the issue because, for example, there has been a problem with the distribution of the issue, most files for the particular report issue will have to be deleted. There is a special script for this pupose

ernad@aigtu:~$ delete_a_nep_report_issue report date

where report is the report code and date is the date of the issue. The script performs sanity checks for the two arguments.

Mailing a report issue again

If there has been a problem with the distribution of an issue, an issue can be mailed out again. There is a special script for this pupose. The script only holds for issues prepared after 2007-03-07, when a special patch to Final.pm ensures a preservation of the mail file.

ernad@aigtu:~$ remail_a_nep_report_issue report date

where report is the report code and date is the date of the issue. The script performs sanity checks for the two arguments.

Setting a new password for a list

The command is

change_pw -l list -p new_password

where list is name of the list and where new_password is the new password.

Setting the general password for all lists

The command is

mmsitepass new_password

where new_password is the new password.

Editing report information

All information about reports is kept in ~/ernad/etc/nep/reports.amf.xml. Before you make any change to that file make an archival copy of the file. Run

ernad@aigtu:~$ archive_reports_file nep

The file documents each report in blocks such as

 <haspart>
  <collection id="RePEc code of nep report">
   <ernad:repcode>code</ernad:repcode>
   <title>subject</title>
   <homepage>http://lists.repec.org/mailman/listinfo/NEP_report_code</homepage>
   <ernad:password>password of editor for ernad</ernad:password>
   <haseditor>
    <person ref="ref">
     <name>name of editor</name>
     <homepage>homepage of editor</homepage>
     <email>email address of editor</email>
     <ispartof>
       <organization>
         <name>name of institution</name>
         <homepage>homepage of institution></homepage>
         <postal>country name only</postal>
       </organization>
      </ispartof>
    </person>
   </haseditor>
  </collection>
 </haspart>

The above should be self-explanatory. The ref is the short-id of the person in the RePEc author service, without the leading “p”.

You have to save the file in UTF-8 encoding when changing it. One way to do this is to use the emacs editor directly on aigtu.

When you have made your changes, check the file with

ernad@aigtu:~/ernad/etc/nep$ xmlwf reports.amf.xml

It this prints out anything, copy the saved version back immediately. If it does not complain, install the documentation with the commands

ernad@aigtu:~$ mainpage nep

and

ernad@aigtu:~$ make_access_pages nep

Note that this will also update the subscriber numbers to the latest found in the data. There should be data from the first of the current month. If this is not there, notify the wizard immediately.

Finally, execute

ernad@aigtu:~$ kill_ernad nep

to have apache restart ernad.

Opening a new report

No person must be allowed to work as an editor unless that person is registered with the RePEc Author Service. At the time that the personal information is added the RAS pages of the person must be added as well. If they are not there, the editor looses credit for editing and may also loose data fed into her pre-sorting information.

When opening a new report, say nep-NEW, you should first agree with the editor the email address email_of_new_editor that she will be using and the Mailman password password_agreed_with_new_editor that she will be using. The general password for NEP should not be used as a password for individual reports.

Once you have the email of the editor, subscribe the address to the nep-editors list. We need to check periodically that all editors are on that list, but that is not done yet.

Next, carefully enter the report information into reports.amf.xml as detailed in the previous section.

Then, execute

ernad@aigtu:~$ kill_ernad nep

to make the new reports.amf.xml file visible to a new copy of ernad.

The next step is to open the list
ernad@aigtu:~$ open_list nep-new
The open_list script creates the list and configures it for you. You still have to do some manual tuning of the list, as outlined in the section “Configuring a list manually”. And please recall that new editors need to be added to the nep-editors list.

Finally, you need to create the access pages for the new reports.

ernad@aigtu:~$ make_access_pages nep

Configuring a list automatically

This is done with the special script "configure_nep_list".

ernad@aigtu:~$ configure_nep_list nep-new

It will say something like
Importing set_nomail...
Running set_nomail.set_nomail()...
Loading list nep-new (unlocked)
Finalizing
Importing set_mod...
Running set_mod.set_mod()...
Loading list nep-new (unlocked)
Finalizing

This output comes from the fix_posting script.

Configuring a list manually

Unfortunately, the automatic configuration at this time does not fix the capitalization. Thus the list created is Nep-new rather than nep-new. You need to get into the admin interfact at http://lists.repec.org/cgi-bin/mailman/admin/nep-new/ to fix this twice. Both changes are in the “General Options” screen. First in the “The public name of this list” and second in “Prefix for subject line of list postings”, you have to turn the letters to lowercase for marketing consistency. Finally, on the same page you need to set “Maximum length in kilobytes (KB) of a message body.” to 0.

Now, it is time to create an access page for the report. There is no specific way for one report, so it’s a general one.

ernad@aigtu:~$ make_access_pages nep

Listing members of a list

This is useful when you want to recreate a list afresh to hopefully cure a problem.

ernad@aigtu:~$ /var/lib/mailman/bin/list_members list
christian.calmes@uqo.ca
gilles.sanou@uemoa.int
jeffrey.robertson@aph.gov.au
library@ote.gr
malarselvam@rediffmail.com
marco@novarese.org
thivant@univ-lyon3.fr

You can save the list in a file for later use

ernad@aigtu:~$ /var/lib/mailman/bin/list_members list > file

where file is the name of the file you want to send the list of members in. When you later add the members back to the re-created list, please note that you should switch off the sending of the welcome message. By default it is switched on. If the members get that message, they will be very confused.

Removing a list

If you want to remove list nep-old, do

ernad@aigtu:~$ rmlist nep-old

If you want to remove list nep-old and its archive.

ernad@aigtu:~$ rmlist -a nep-old

Removing a member

This can be done on the command line.
ernad@aigtu:~$ remove_members list email
where list is a list and email is the email of the member to be removed. Example
ernad@aigtu:~$ remove_members nep-all S.R.Epstein@lse.ac.uk
The special syntax --fromall can be used to
ernad@aigtu:~$ remove_members --fromall email
to remove a member with email email from all lists. Example
ernad@aigtu:~$ remove_members --fromall S.R.Epstein@lse.ac.uk

Checking posting permissions

Normally, all members are moderated. ernad at nep repec org is a member of the list. It is the only member that is not moderated.

In olden days ernad at nep repec org was not a member of the list. But it was set to be allowed to post to the list as a non-member. This works for most lists. If it fails, ernad at nep repec org is informed with a standard text "only the list owner can post to this list". In that case it appears necessary to admit ernad at nep repec org as a member and then remove the moderation flag for this member. This is now the standard configuration. To implement this on a list use
ernad@aigtu:~$ fix_posting list
where list is the list to be fixed. It will say something like
Importing set_nomail...
Running set_nomail.set_nomail()...
Loading list list (unlocked)
Finalizing
Importing set_mod...
Running set_mod.set_mod()...
Loading list list (unlocked)
Finalizing

Use of putty

Here is how you should proceed to login to nep.repec.org.

Customizing putty

  1. Open putty

    You are now in the Sessions screen of putty.
  2. In the Category menu, choose Translation

    You are now in the Options controlling character set translation screen of putty.
  3. In the Received data assumed to be in which character setdrop down box, choose UTF-8

  4. In the Category menu, choose Sessions

    You are now back in the Session screen of putty.
  5. Click on Default Settings

  6. Click on Save

  7. Set Host Name (or IP address) to nep.repec.org

  8. Set the Protocol to SSH

  9. Set Host Name (or IP address) to nep.repec.org

  10. Set the Saved Sessions to nep.repec.org

  11. Click on Save

  12. Exit putty

Using putty

  1. Open putty

    You are now in the Session screen of putty.
  2. Double-click nep.repec.org

  3. At the Login prompt, enter ernad

  4. At the Password prompt, enter the password

Managing the mailing list

They are