Understanding File Attachments and PeopleCode

This chapter discuses using file attachments in applications and PeopleCode.

Using File Attachments in Applications

This section provides an overview of file attachment architecture and discusses how to:

Debug file attachment problems.
Configure multiple application servers to support file attachments.
View file attachments.
Use chunking with attachments.
Use the Microsoft Windows NT 4 FTP server.
Use non-DNS URLs with UNIX.
Use attachment functions.
Name URLs and file attachments.
Convert user filenames.

Understanding File Attachment Architecture

File attachments are supported by using PeopleCode built-in functions that implement the transfer of a file to or from a browser using the application server. The file is either stored on, or retrieved from, a file server or database tables, referred to as the data storage system.

Files can be transferred back and forth from the browser to the web server, to the application server, to the data storage system.

The browser-web server transfer is performed using a standard HTML form construct. This can be done securely in an encrypted fashion if the web server uses Secure Sockets Layer (SSL) to communicate to the browser. After the file is received at the web server, 1 megabytes chunks are brought from the web server to the application server and stored to the database.

Note. The 1 megabyte transfer size is not customizable.

Once the entire file is transferred, the application server reassembles the file and sends it to either a FTP server or to a database table, depending. After the transfer is complete, the file is deleted from the database.

Note. The MaxSize variable is checked before the file is transferred to the web server. If the file exceeds the value specified with the MaxSize parameter, it is not transferred, and the system issues an error message.

The web server-application server transfer is performed by using BEA Jolt, which is securely encrypted. Because this transfer is done using the standard BEA Jolt mechanism, no additional settings to the firewall are required (there's no need to open additional ports). On the application server, the file is stored in one of the following temporary directories, depending on the operation:

/tmp/PSFTP/ATransactionNumber (for attach operation)
/tmp/PSFTP/VTransactionNumber(for view operation)

TransactionNumber is randomly generated for both attaching and viewing.

By using separately named subdirectories for each attachment, support for the simultaneous transfer of identically named files is guaranteed, and attachments won't be mixed up.

The application server-data storage system transfer is performed either using File Transfer Protocol (FTP) or directly to a database table.

The FTP transfer can be securely encrypted. The FTP service is invoked on the application server by a script that is passed to the FTP binary. This script is in /tmp and is called ftpNumber.txt. The output of invoking the script is called ftpNumber.log (The number variable is a random number.) These files are left in /tmp for debugging, with the assumption that access to the application server is restricted and somewhat secure. For your own security, you may want to delete these files on a weekly basis.

Note. You cannot specify a different port for an attachment when using an FTP server. You can only use the default port of 21.

No log file or script is required for writing to database tables. For debugging, these commands leave traces in the PeopleCode trace files if Internal Tracing (256) is selected.

Note. The search API does not work with database attachments.

Debugging File Attachment Problems

To verify that the FTP transfer was successful, access the application server and examine the ftpNumber_Number_Number.log file and the ftpNumber_Number_Number.txt file.

The following is an example of a log file. It shows that the file transfer was successful.

pt-hp01:$ cat ftp1039.txt open 216.131.219.51 user ANONYMOUS TEST bin lcd / lcd tmp lcd PSFTP lcd V13333 cd / get "IA_FA_Test.TXT" "IA_FA_Test.TXT" quit pt-hp01:$ pt-hp01:$ cat ftp1039.log about to submit ftp -vin /tmp/ftp1039.txt Connected to 216.131.219.51. 220 TCARREON052500 Microsoft FTP Service (Version 4.0). 331 Anonymous access allowed, send identity (e-mail name) as password. 230 Anonymous user logged in. 200 Type set to I. Local directory now / Local directory now /tmp Local directory now /tmp/PSFTP Local directory now /tmp/PSFTP/V13333 250 CWD command successful. 200 PORT command successful. 150 Opening BINARY mode data connection for IA_FA_Test.TXT(72 bytes). 226 Transfer complete. 72 bytes received in 0.06 seconds (1.21 Kbytes/s) 221 File Transfer Successful pt-hp01:$

If the file transfer wasn't successful, error messages in the log can help you determine the problem.

A common reason that a transfer fails is that the FTP server was not accessible from the application server. This could be due to a wrong password or wrong account name, or perhaps because the application server was unable to resolve the FTP server hostname. Perhaps the FTP server was down and the application server was unable to route to it. Try pinging the FTP server from the application server system, and then try to FTP to the FTP server from the application server.

If you have a Microsoft Windows NT file server, it's possible that the hostname for the NT system is not associated with a fixed IP address and is not resolvable using the Domain Name System (DNS). If you have the application server on a UNIX machine (such as Solaris, HP-UX, or AIX), the application server can only resolve the hostname using DNS (or perhaps the Network Information System [NIS] or an /etc/hosts file), but not using WinBeui or WINS. This means that the application server won't be able to convert the hostname indicated for the Microsoft Windows NT file server into an IP address and route to it.

If you suspect this is the problem, try the following:

Telnet to the UNIX application server.
Try to ping the suspect hostname.

For example:
```
ping TCARREON052500
```

If this fails, you must resolve the problem by either specifying the IP address in the FTP URL or by putting your hostname into DNS, NIS, or a hosts file so that UNIX can resolve it.

Debugging Database Attachments

Turn on tracing for internal PeopleCode command using either the SetTracePC function with the value %TracePC_IntFuncs (256), or the trace functions used with PeopleSoft Pure Internet Architecture, to generate information about each step of the attachment process for database attachments.

Debugging Paths

In general, FTP servers require the full path from the top-level directory, rather than the path from the home directory (a relative path). If you've been able to add an attachment but can't view it, verify that you're using a full, absolute path, as opposed to a relative path.

Attachments with Double-Byte Character Filenames

To successfully upload attachments whose filename comes from a language which uses a double-byte character string, such as Japanese, from your web browser to either an FTP server or database table, the application server must be running in an environment that supports double-byte character languages.

If the destination for your attachment is an FTP server, that FTP server must also be running in an environment that supports double-byte character languages. (The web server can be running on either an English environment or a double-byte character language environment.)

Configuring Multiple Application Servers to Support File Attachments

When you have multiple application servers, the /tmp/PSFTP/DomainIDxxx directory is created by the first application server process to run on your application server. The assumption is that all such application server processes run with the same user ID. However, a single computer might run application servers with many user IDs.

The default /tmp/PSFTP is created so that all users can access it (777 permissions, or drwxrwxrwx). For proper security, however, the DomainIDxxx directory is created so that only that application server can access that directory (711 permissions, or drwx--x--x). If multiple users need read and write access, you must change the permissions.

When using file objects, the service request can be sent to different application servers, so that if a file was created under one application server machine environment and that service is next serviced on a different application server machine, the file might not exist. Therefore, your application should check that the file is open, particularly when returning from a user interraction (since the process of the handling user interface results in a serialization/deserialization of all the transaction states.)

Viewing File Attachments

When viewing a file attachment, the browser invokes a viewer based on the content-type Multipurpose Internet Mail Extensions (MIME) category sent in the response header from the web server.

For example, if the user tried to view an MP3 file, the response header sent to the browser by the web server would indicate the audio/MPEG content type.

HTTP/1.1 200 OK Server: Microsoft-IIS/5.0 Date: Mon, 01 Oct 2001 21:25:51 GMT Content-Type: audio/mpeg Accept-Ranges: bytes Last-Modified: Mon, 01 Oct 2001 21:00:26 GMT ETag: "78e21918bc4ac11:cc8" Content-Length: 60

Notice that the Content-Type is audio/mpeg. The browser uses this MIME type to determine that a viewer for audio/MPEG is appropriate. If the web server did not send this content-type header, the browser wouldn't be able to determine the nature of the file being transmitted, and would be unable to invoke the correct application. The browser would try to display the file as text/plain. This might be the wrong assumption.

The web server determines the MIME type by looking at the extension of the attachment file and mapping it to a MIME type. The mapping is done in the following ways, depending on the web server and platform:

For Microsoft IIS, the mapping is done by using the standard Microsoft Windows extension association mechanism.
For Apache, the mapping is done by editing a configuration file called mime.types.
For WebLogic, a file called web.xml does the mapping.

The file web.xml is located in the WEB-INF subdirectory. This file contains a section that looks like this:

  </mime-mapping> 
  <mime-mapping> 
    <extension> 
      doc 
    </extension> 
    <mime-type> 
       application/msword 
    </mime-type> 
  </mime-mapping> 
  <mime-mapping> 
    <extension> 
      xls 
    </extension> 
    <mime-type> 
       application/vnd.ms-excel 
    </mime-type> 
  </mime-mapping>

Let's say you want to add an extension that causes log files to be interpreted as plain text files.

To determine the correct MIME type, check Request for Comments (RFC) documents 2045, 2046, 2047, 2048, and 2077, which discuss internet media types and the internet media type registry.

After checking the RFCs, you determine that the correct MIME type is text/plain. The following is an example of you would add to the above section of web.xml:

  </mime-mapping> 
  <mime-mapping> 
    <extension> 
      doc 
    </extension> 
    <mime-type> 
       application/msword 
    </mime-type> 
  </mime-mapping> 
  <mime-mapping> 
    <extension> 
      log 
    </extension> 
    <mime-type> 
       text/plain 
    </mime-type> 
  </mime-mapping> 
  <mime-mapping> 
    <extension> 
      xls 
    </extension> 
    <mime-type> 
       application/vnd.ms-excel 
    </mime-type> 
  </mime-mapping>

Once you save the file, the log extension is now associated with the content type of text/plain.

Note. You must restart the WebLogic server before these changes are recognized.

Similar mappings can be achieved on other web servers, such as IBM WebSphere or Netscape Enterprise Server.

See your web server documentation.

Note. When trying to view the objects, the extension must match exactly what is set up in the web.xml file. This value is case-sensitive. If the object view appears garbled, chances are that either the extension is not set up in the web.xml file, or there is a case mismatch.

Using Chunking with Attachments

When writing data to database tables, the data is automatically chunked, or stored in different rows of the database table. The size of each chunk is determined by the Max Chunk File Size field on the PSOPTIONS page. The default value of this field is 28,000 kilobytes (K).

Because each file is chunked, you can't directly pull data from the table. You should use the provided attachment functions, which automatically put the data back together into one file for you. Because the chunk size is stored with the file, if you change the system chunk size you can still retrieve files with different chunk sizes.

DB2 UDB for OS/390 and z/OS Considerations

If you store file attachments on DB2 UDB for OS/390 and z/OS, you're limited to chunk sizes of 3862K for a 4K tablespace, or 32444K for a 32K page tablespace. By default, the only tablespace with 32K is PSIMAGE.

Using the Microsoft Windows NT 4 FTP Server

If you're using FTP, and running Microsoft Windows NT 4 with the standard Microsoft FTP server, you must take the following into consideration.

Non-Microsoft FTP server programs on Microsoft Windows NT, the Microsoft Windows 2000 FTP server, as well as all UNIX FTP servers, execute a chroot command to the home directory of the FTP server automatically.

However, the Windows NT 4 FTP server does not follow this behavior. When an FTP client connects to the Microsoft FTP server, the client can gain access to the entire drive, not just the FTP home directory.

For example, suppose you are running the Microsoft Windows NT 4 FTP server and the home directory is c:\inetpub\ftproot.

When you log in, your current directory is c:\inetpub\ftproot.

When you type the following:

cd \

Your current directory is now c:\.

If you are not running the Microsoft Windows NT 4 FTP server, and the home directory is the same: c:\inetpub\ftproot.

When you type the following:

cd \

Your current directory is now c:\inetpub\ftproot.

This means that you must know the FTP server associated with the URL.

If you are using a Windows NT 4 FTP server, or another FTP server that shows this behavior, your URLs should have the following format: ftp://ftp:ftp@NT4system/inetpub/ftproot/tmp. Note that the home directory of the FTP server must be appended to the address.

If you are not using these sorts of servers, your URLs should have the following format: ftp://ftp:ftp@NT4system/tmp.

Using Non-DNS URLs with UNIX

The URL passed to file attachments generally has the following format: ftp://user:pass@systemname/dir1/dir2.

This assumes that the domain name systemName can be resolved with DNS.

Some Microsoft Windows systems can resolve systemName even when it can't be resolved with DNS. However, some UNIX systems cannot these resolve names.

If you are using a UNIX system, and the domain name cannot be resolved with DNS, use the IP address. The following example assumes systemName has the IP address of 123.123.123.123: ftp://user:[email protected]/dir1/dir2.

To determine the IP address, you can try to ping the system by typing the following:

ping systemName

You should use domain names that can be resolved. Only use IP addresses when absolutely necessary.

Using Attachment Functions

To store attachments in a database, include the FILE_ATTACH_SBR subrecord and the FILE_ATTACH_WRK work record in your component when using the file attachment functions.

Note. You do not have to include the FILE_ATTACH_SBR subrecord in your component; you could just include the fields. However, it's better to include the subrecord.

The FILE_ATTACH_SBR subrecord contains the following fields:

Field	Description
ATTACHSYSFILENAME	The system file name.
ATTACHUSERFILE	The user file name.

No PeopleCode is associated with this subrecord. Developers should include this subrecord in target records for using attachments.

The FILE_ATTACH_WRK work record contains the following fields:

Field	Description
ATTACHADD	Contains a PeopleCode program used for adding attachments (the AddAttachment built-in function).
ATTACHDELETE	Contains a PeopleCode program used for deleting attachments (the DeleteAttachment built-in function).
ATTACHVIEW	Contains a PeopleCode program used for viewing attachments (the ViewAttachment built-in function).

The PeopleCode program supplied by PeopleSoft with this record is associated with the FieldChange events for each function, and the RowInit event for the ATTACHADD function. Because the PeopleCode is in the form of functions, this work record acts as a container for the appropriate methods related to attachments. Instead of including this work record in your component, you can use your own work record and call these functions as needed from within your work record.

For storing attachments in the database, you must create a record that receives the attachments. This record can be called anything, but it must have the FILE_ATTDET_SBR subrecord included in it, and have no additional fields in it. The subrecord FILE_ATTDET_SBR has the following fields:

Field	Description
ATTACHSYSFILENAME	A unique system file name.
FILE_SEQ	The file sequence number (used in chunking).
VERSION	Version number.
FILE_SIZE	The physical size of the file.
LASTUPDDTTM	Last update date and time.
LASTUPDOPRID	The employee ID of the last user to update the attachment.
FILE_DATA	The data of the file.

To use the attachment functions:

Insert the subrecord FILE_ATTACH_SBR into a record used with your page.

Typically, this record is inserted as part of the primary database record.
To store the file in the database, insert the subrecord FILE_ATTDET_SBR into a record used with your page.

This record can be called anything, but it must have the FILE_ATTDET_SBR subrecord included in it, and have no additional fields in it. This is the record that you point to with the URL when specifying storing a file in a database table.
Add buttons to add, delete, and view attachments as needed.

You can either use the FILE_ATTACH_WRK record for the necessary record fields, or you can use your own work fields.
Set up the URL.

If you're using the FTP archive, suppose the system is fileserver.ps.com, the user account is joe, and the password is secret. The full URL is: ftp://joe:[email protected]/.

You can either create an entry for this URL in the URL Maintenance page, or specify it in your PeopleCode.

If you specify the URL in your PeopleCode, you don't have to hard-code the username and password. You can dereference them using a variable or record field. This enables you to use encryption with the username and password.

If you're copying to a database table, you can either create an entry for this record (URL) in the URL Maintenance page, or specify it in your PeopleCode, as follows:
```
record://RESUMES_DB
```
Modify the PeopleCode to support your application.

If you use the supplied PeopleCode programs, you must specify the URL. You may also want to modify the extension of the expected files (the default is "", which is any file, or *.* You may want to indicate ".doc", ".xl", "*.html", "*.exe", and "*.tar".). Read the comments in ATTACHADD FieldChange program for instructions.

To use the FILE_ATTACH_WRK record for buttons, you may want to use component PeopleCode for your application-specific code. This enables you to keep the code in FILE_ATTACH_WRK generic. If you use your own work fields, you can write your own code and call the functions in FILE_ATTACH_WRK as needed.

See URL Maintenance.

Coding Considerations Using the Attachment Functions

All file attachments are performed using PeopleCode built-in functions, such as AddAttachment, ViewAttachment, GetAttachment, and so on. These functions move a file into and out of a file storage system, either the database or a file system. The file system movement is done using the FTP protocol.

Because these functions abstract the storage of the attachments, you can use either database or FTP file systems. The system to be used is determined by the URL passed as the first parameter in the attachment built-in function. The actual value of the URL is maintained on the URL Maintenance page. This means that the you don't know whether users will store attachments in a database or a file system.

The following PeopleCode functions are used with administration work:

DeleteAttachment deletes unwanted attachments.

There is no roll-back ability. For example, suppose a user selects to view an attachment, then cancels before the attachment is displayed. A copy of the attachment may still be in the user's temp directory.
CopyAttachments moves all of the attachments specified from one file storage system (database or FTP) to another (database or FTP).
CleanAttachments is an audit-like script that removes attachments that exist in the database that are not referenced by any record field.

If you only have a few files to copy using CopyAttachments, you may want to use the CopyFiles page (part of the PeopleTools Administration pages). The CleanAttachments function is also available from this page.

To schedule a regular job to clean up orphaned file attachments, you can use the Application Engine program CLEANATT84.

Store all references to attachments using the standard ATTACHSYSFILENAME field. Do not reuse this field to store incomplete or nonstandard versions of the name. For example, you should not store the full URL version of the attachment in this column and then use PeopleCode to parse the URL before invoking attachment commands. While this may work for get or put attachment calls, the Clean Attachments function deletes any file stored in a table that does not have a corresponding reference stored in ATTACHSYSFILENAME.

Copying Files Using the File Attachment Functions

The CopyAttachments function is used for moving a group of attachments from one archive to another, such as when you want to move files from one FTP server to another, or if you want to move all the files in a database record out onto an FTP server. It does not create new system file links.

If you want to create a new file attachment by making a copy of an existing file attachment (both the system file link and the file itself, ) do the following:

Use GetAttachmentto retrieve the file from the repository (either from the FTP server or from a database record) and place it on your application server (or process scheduler system if you are using PeopleSoft Application Engine.)
Use PutAttachment to move the file from the application server (or process scheduler system) and place it into the new archive location.
Save the new system file link that you passed as a parameter to PutAttachment so that you can subsequently use it. This is generally part of the page transaction when you save the record that has the FILE_ATTACH_SBR subrecord in it.

Naming URLs and File Attachments

If you change a URL on the URL Maintenance page from FTP to record, you may have to reboot your application server before you can use the URL in your PeopleCode program.

Converting User Filenames

If a user attaches a file through the browser dialog box generated by the AddAttachment function, the filename ay be converted before that filename is returned to the AddAttachment call for storage.

For example, the file My Resume.doc is returned through the AddAttachment parameter as My_Resume.doc, with the space changed to an underscore.

No comparable mapping occurs for user filenames with the PutAttachment function, which requires the user filename as an input parameter, rather than an output parameter that is returned to the user.

Instead, before storing the ATTACHUSERFILE value, code similar to this should be invoked:

   &ATTACHUSERFILE = Substitute(&ATTACHUSERFILE, " ", "_"); 
   &ATTACHUSERFILE = Substitute(&ATTACHUSERFILE, ";", "_"); 
   &ATTACHUSERFILE = Substitute(&ATTACHUSERFILE, "+", "_"); 
   &ATTACHUSERFILE = Substitute(&ATTACHUSERFILE, "%", "_"); 
   &ATTACHUSERFILE = Substitute(&ATTACHUSERFILE, "&", "_"); 
   &ATTACHUSERFILE = Substitute(&ATTACHUSERFILE, "'", "_"); 
   &ATTACHUSERFILE = Substitute(&ATTACHUSERFILE, "!", "_"); 
   &ATTACHUSERFILE = Substitute(&ATTACHUSERFILE, "@", "_"); 
   &ATTACHUSERFILE = Substitute(&ATTACHUSERFILE, "#", "_"); 
   &ATTACHUSERFILE = Substitute(&ATTACHUSERFILE, "$", "_");