• Turns out my netcdf libraries under /usr/local/OPeNDAP*/lib had 32 bit objects, but not 64 bit objects, so I built a 64 bit version, and merge-libs'd them.

• Weird. dap_nc_handler doesn't build with gcc/g++ all of a sudden, and it appears that there is no /usr/local/OPeNDAP*/lib/libdap.a. There is one under /usr/local/lib though.
• So I'm building one - under /usr/local/OPeNDAP*/lib.

• Output from dap_nc_handler built with xlc:

  esmf04m-strombrg> ./dap_nc_handler -v DAP2/3.5.1 -o dds -u http://esmf.ess.uci.edu/cgi-bin/dods/nph-dods/dodsdata/in.nc /usr/local/apache-2.0.55/htdocs/dodsdata/in.nc -r /usr/tmp
  HTTP/1.0 200 OK
  XDODS-Server: 3.5.2
  Date: Wed, 08 Feb 2006 22:28:13 GMT
  Last-Modified: Wed, 08 Feb 2006 22:28:13 GMT
  Content-type: text/plain
  Content-Description: dods_error
  Cache-Control: no-cache
  
  Error {
      code = 1002;
      message = "An internal error was encountered:
  Call to set_parent with incorrect variable type.
  Please report this to support@unidata.ucar.edu
  ";
  };
  Wed Feb 08 14:28:13
  

• Output from dap_nc_handler built with gcc (But see above for the process to recreate this):

• Unfortunately, we're getting the same error at UCAR that we have been on the ESMF. The idea behind building at UCAR was to see if the problem would go away with IBM XL C/C++ 7.0.0.2. We still get:

• bash-3.00$ ./dap_nc_handler -v DAP2/3.5.1 -o dds -u http://esmf.ess.uci.edu/cgi-bin/dods/nph-dods/dodsdata/in.nc ~/lib/in.nc -r /usr/tmp
  HTTP/1.0 200 OK
  XDODS-Server: 3.5.2
  Date: Tue, 07 Feb 2006 01:33:45 GMT
  Last-Modified: Tue, 07 Feb 2006 01:33:45 GMT
  Content-type: text/plain
  Content-Description: dods_error
  Cache-Control: no-cache
  
  Error {
      code = 1002;
      message = "An internal error was encountered:
  Call to set_parent with incorrect variable type.
  Please report this to support@unidata.ucar.edu
  ";
  };
  bash-3.00$ 
  

esmf04m-root> dbx ../../netcdf-handler/src/dap_nc_handler 
Type 'help' for help.
reading symbolic information ...
(dbx) help file
file
file <filename>
        Change the current source file name to <filename>. If
        none is specified then the current source file name is
        printed.
(dbx) file InternalErr.cc
(dbx) stop in InternalErr::InternalErr
1. InternalErr.InternalErr::InternalErr(const InternalErr&)
2. InternalErr.InternalErr::InternalErr()
3. InternalErr.InternalErr::InternalErr(std::basic_string<char,std::char_traits<char>,std::allocator<char> >,ProgramType,char*)
4. InternalErr.InternalErr::InternalErr(const std::basic_string<char,std::char_traits<char>,std::allocator<char> >&,const int&,const std::basic_string<char,std::char_traits<char>,std::allocator<char> >&)
5. InternalErr.InternalErr::InternalErr(const std::basic_string<char,std::char_traits<char>,std::allocator<char> >&)
Select one or more of [1 - 5]: 1,2,3,4,5
[1] stop in InternalErr.InternalErr::InternalErr(const InternalErr&)
[2] stop in InternalErr.InternalErr::InternalErr()
[3] stop in InternalErr.InternalErr::InternalErr(std::basic_string<char,std::char_traits<char>,std::allocator<char> >,ProgramType,char*)
[4] stop in InternalErr.InternalErr::InternalErr(const std::basic_string<char,std::char_traits<char>,std::allocator<char> >&,const int&,const std::basic_string<char,std::char_traits<char>,std::allocator<char> >&)
[5] stop in InternalErr.InternalErr::InternalErr(const std::basic_string<char,std::char_traits<char>,std::allocator<char> >&)
(dbx) run -v DAP2/3.5.1 -o dds -u http://esmf.ess.uci.edu/cgi-bin/dods/nph-dods/dodsdata/in.nc /usr/local/apache-2.0.55/htdocs/dodsdata/in.nc -r /usr/tmp
[5] stopped in InternalErr.InternalErr::InternalErr(const std::basic_string<char,std::char_traits<char>,std::allocator<char> >&) at line 55 in file "InternalErr.cc" ($t1)
   55   InternalErr::InternalErr(const string &msg) : Error()
(dbx) list 50,60
   50   InternalErr::InternalErr() : Error()
   51   {
   52       _error_code=internal_error;
   53   }
   54   
   55   InternalErr::InternalErr(const string &msg) : Error()
   56   {
   57       _error_code=internal_error;
   58       _error_message="";
   59       _error_message+="An internal error was encountered:\n";
   60       _error_message+=msg+"\n";
(dbx) where
InternalErr.InternalErr::InternalErr(const std::basic_string<char,std::char_traits<char>,std::allocator<char> >&)(this = 0x0ffffffffff60510, msg = &(...)), line 55 in "InternalErr.cc"
unnamed block $b2885, line 501 in "BaseType.cc"
unnamed block $b2884, line 501 in "BaseType.cc"
set_parent(BaseType*)(0x110223390, 0x110223230), line 501 in "BaseType.cc"
unnamed block $b3163, line 897 in "Vector.cc"
unnamed block $b3161, line 897 in "Vector.cc"
add_var(BaseType*,Part)(0x110223230, 0x110223130, 0x0), line 897 in "Vector.cc"
add_var(BaseType*,Part)(0x110223230, 0x110223130, 0x0), line 173 in "Array.cc"
unnamed block $b190, line 228 in "ncdds.cc"
unnamed block $b188, line 228 in "ncdds.cc"
read_class(DDS&,int,int,std::basic_string<char,std::char_traits<char>,std::allocator<char> >*)(0xfffffffffffefe8, 0x300000003, 0x1600000016, 0x9000000014f7eb0), line 228 in "ncdds.cc"
unnamed block $b201, line 304 in "ncdds.cc"
read_descriptors(DDS&,const std::basic_string<char,std::char_traits<char>,std::allocator<char> >&)(0xfffffffffffefe8, 0xffffffffffff110), line 304 in "ncdds.cc"
unnamed block $b216, line 71 in "nc_handler.cc"
unnamed block $b213, line 71 in "nc_handler.cc"
main(argc = 10, argv = 0x0ffffffffffff668), line 71 in "nc_handler.cc"
(dbx) dump
InternalErr.InternalErr::InternalErr(const std::basic_string<char,std::char_traits<char>,std::allocator<char> >&)(this = 0x0ffffffffff60510, msg =               _String_base:()
          _String_val<char,std::allocator<char> >:(_Alval = ())
      (_Ptr = "Call to set_parent with incorrect variable type.", _Len = 48, _Res = 63)), line 55 in "InternalErr.cc"
(dbx) 

• And here are some unit test results - they didn't fare too well.

• I rebuilt the netcdf handler in response to an e-mail session between Harry and Yuan - we're now using netcdf_handler-3.5.2.tar.gz, built with xlc (cobbled, of course).
• As I'm truss'ing the ESMF's apaches during:

  esmf04m-strombrg> getdap -d http://esmf.ess.uci.edu/cgi-bin/dods/nph-dods/dodsdata/in.nc.dds
  "This DODS server does not know how to serve the dataset
  `/dodsdata/in.nc.dds.'
  It maybe that the server has not been configured for this type of dataset.
  Please double check the URL for errors and, if you think that the URL is
  correct, please contact the administrator of this site at: dcs\@uci.edu"
  Fri Jan 13 16:10:51
  

  ...I'm noticing that when nph-dods is exec'd by apache, nph-dods is doing some pretty serious I/O against /unix! Could it be looking for a loadaverage? Which other legitimate purpose would a CGI script have to do kernel groveling?
• That interesting /unix I/O session is followed by 3088396: loadquery(0x00000002, 0x20227888, 0x00005000, 0x000010B0, 0x00000000, 0x00000000, 0xF0097274, 0x00000000) = 0x00000000
• It's getting perl modules from the system stuff:

  3088396: statx("/usr/opt/perl5/lib/5.6.0/aix/lib.pmc", 0x2FF21F80, 128, 010) Err#2  ENOENT
  3088396: open("/usr/opt/perl5/lib/5.6.0/aix/lib.pm", 0400000000) Err#2  ENOENT
  3088396: statx("/usr/opt/perl5/lib/5.6.0/lib.pmc", 0x2FF21F80, 128, 010) Err#2  ENOENT
  3088396: open("/usr/opt/perl5/lib/5.6.0/lib.pm", 0400000000) = 4
  

• Appears to be a useful configuration file:

  3088396: open("/usr/local/share/dap-server/read_config.pm", 0400000000) = 5
  

• Appears to be another useful configuration file:

  3088396: open("./dap-server.rc", 0400000000)    = 3 
  

• I've added a couple of lines to our dap-server.rc:

  handler .*\.(DDS|dds)$ dap_nc_handler -O dds
  curl /usr/local/bin/curl
  

• Just noticed that this isn't working either, although it used to:

  esmf04m-strombrg> getdap -d http://esmf.ess.uci.edu/cgi-bin/dods/nph-dods/dodsdata/in.nc
  "An internal error was encountered:
  Call to set_parent with incorrect variable type.
  Please report this to support@unidata.ucar.edu
  "
  Fri Jan 13 16:50:44
  

• Truss'ing is at least showing that we're getting into the netcdf handler. I should do the next truss with a -a to see what those 10 arguments are:

  1646676: execve("/usr/local/bin/dap_nc_handler", 0x203484C8, 0x20233B28)  argc: 10
  

• Shortly after this, the netcdf handler is terminated, and this message suddenly appears in the busiest apache process, without any apparent system call associated with it:

  2810110:            c o d e   =   1 0 0 2 ;\n         m e s s a g e   =   "
  2810110:    A n   i n t e r n a l   e r r o r   w a s   e n c o u n t e r e
  2810110:    d :\n C a l l   t o   s e t _ p a r e n t   w i t h   i n c o r
  2810110:    r e c t   v a r i a b l e   t y p e .\n P l e a s e   r e p o r
  2810110:    t   t h i s   t o   s u p p o r t @ u n i d a t a . u c a r . e
  2810110:    d u\n " ;\n
  

  ...but it might be coming from this, which is the closest syscall from 2810110:

  2810110: kwritev(8, 0x2FF20040, 1, 0x00000000)1646676:   unamex(0x0000000000000002, 0x0000000000000000, 0x000000011020EF70, 0x900000000000D0B2, 0x0000000000000000, 0x0000000000000000, 0x0000000000000000, 0x0000000000000000)     = 8 = 0x0000000000000005
  

• Truss with -a reveals:

  2793660: execve("/usr/local/bin/dap_nc_handler", 0x203484C8, 0x20233B28)  argc: 10
  2793660:  argv: dap_nc_handler -v DAP2/3.5.1 -o dds -u
  2793660:   http://esmf.ess.uci.edu/cgi-bin/dods/nph-dods/dodsdata/in.nc
  2793660:   /usr/local/apache-2.0.55/htdocs/dodsdata/in.nc -r /usr/tmp
  

• Nice replication (we've been here before, in a sense, but not with these precision options):

  esmf04m-root> dap_nc_handler -v DAP2/3.5.1 -o dds -u http://esmf.ess.uci.edu/cgi-bin/dods/nph-dods/dodsdata/in.nc /usr/local/apache-2.0.55/htdocs/dodsdata/in.nc -r /usr/tmp
  HTTP/1.0 200 OK
  XDODS-Server: 3.5.2
  Date: Sat, 14 Jan 2006 01:14:16 GMT
  Last-Modified: Sat, 14 Jan 2006 01:14:16 GMT
  Content-type: text/plain
  Content-Description: dods_error
  Cache-Control: no-cache
  
  Error {
      code = 1002;
      message = "An internal error was encountered:
  Call to set_parent with incorrect variable type.
  Please report this to support@unidata.ucar.edu
  ";
  };
  Fri Jan 13 17:14:16
  

I can run your example OK:

        esmf04m-strombrg> getdap -d http://test.opendap.org/dap/data/nc/fnoc1.nc
        Dataset {
            Int16 u[time_a = 16][lat = 17][lon = 21];
            Int16 v[time_a = 16][lat = 17][lon = 21];
            Float32 lat[lat = 17];
            Float32 lon[lon = 21];
            Float32 time[time = 16];
        } fnoc1.nc;
        Wed Jan 04 16:02:44


...but I cannot get a test .nc.dds on our DODS server:

        esmf04m-strombrg> getdap -d http://esmf.ess.uci.edu/cgi-bin/dods/nph-dods/dodsdata/in.nc.dds
        "This DODS server does not know how to serve the dataset
        `/dodsdata/in.nc.dds.'
        It maybe that the server has not been configured for this type of dataset.
        Please double check the URL for errors and, if you think that the URL is
        correct, please contact the administrator of this site at: dcs\@uci.edu"
        Wed Jan 04 16:07:45


Could this be a clue?  Please note the attempts to
retrieve .nc.dds.dds :

        esmf04m-root> for pid in $(ps -ef | grep '[/]usr/local/apache-2.0.55/bin/httpd' | awk ' { print $2 }'); do echo pid is $pid; truss -f -o "$pid.out" -p "$pid" & done
        pid is 819210
        [1] 2809978
        pid is 2072590
        [2] 2302028
        pid is 2990220
        [3] 221390
        Wed Jan 04 16:07:38
        
        esmf04m-root> 
        Wed Jan 04 16:07:39
        
        esmf04m-root> 
        Wed Jan 04 16:07:39
        
        esmf04m-root> d
        
        ./  ../  2072590.out  2990220.out  819210.out
        
        /var/tmp/dods-trussing
        Wed Jan 04 16:08:08
        
        esmf04m-root> fg
        truss -f -o "$pid.out" -p "$pid"
        ^CPstatus: process is not stopped
        Wed Jan 04 16:08:10
        
        esmf04m-root> fg
        truss -f -o "$pid.out" -p "$pid"
        ^CPstatus: process is not stopped
        Wed Jan 04 16:08:10
        
        esmf04m-root> fg
        truss -f -o "$pid.out" -p "$pid"
        ^CPstatus: process is not stopped
        Wed Jan 04 16:08:12
        
        esmf04m-root> d
        
        ./  ../  2072590.out  2990220.out  819210.out
        
        /var/tmp/dods-trussing
        Wed Jan 04 16:08:17
        
        esmf04m-root> grep in.nc *
        2990220.out:2990220:    statx("/usr/local/apache-2.0.55/cgi-bin/dods/nph-dods/dodsdata/in.nc.dds.dds", 0x2FF22090, 76, 0) Err#20 ENOTDIR
        2990220.out:2990220:    statx("/usr/local/apache-2.0.55/htdocs/dodsdata/in.nc.dds.dds", 0x2FF1FDB0, 76, 0) Err#2  ENOENT
        2990220.out:2990220:    statx("/usr/local/apache-2.0.55/htdocs/dodsdata/in.nc.dds.dds", 0x2FF1FD60, 76, 01) Err#2  ENOENT
        2990220.out:2646134:    statx("/usr/local/apache-2.0.55/htdocs/dodsdata/in.nc.dds.dds", 0x2001C030, 128, 010) Err#2  ENOENT
        Wed Jan 04 16:08:19
        
        esmf04m-root> d /usr/local/apache-2.0.55/htdocs/dodsdata/
        in.nc     testfile  
        Wed Jan 04 16:08:19
        


Here's our dods.rc:

        esmf04m-root> cat /usr/local/apache-2.0.55/cgi-bin/dods/dods.rc 
        # -*- Perl -*-
        # OPeNDAP server configuration file.
        # Blank lines are ignored; comments must start with `#' in the first column.
        
        # Set the server time out. This is different from the httpd time out (which
        # does not apply to an OPeNDAP server). The value is in seconds; a value of
        # zero disables the time out feature.
        timeout 0
        
        # Compressed files are decompressed first before serving. Where should these
        # decompressed files be cached?
        cache_dir /usr/tmp
        
        # How big a cache? If you're serving some large files, make sure this is at
        # least big enough to hold one of them! The size is given in MegaBytes.
        cache_size 500
        
        # Email address of the person in charge of this server. 
        maintainer dcs@uci.edu
        
        # List of handlers whose regular expressions should not be used when building
        # the HTML interface to this server. This includes any overly inclusive
        # regular expression like '.*' which matches anything. This will make README
        # files, et.c, show up as datasets. Typically only the JGOFS server every has
        # these regular expressions. Each handler name should be quoted, separated by
        # whitespace (not a comma).
        #exclude jg
        
        # Match various handlers with pathnames. In *most* cases this means filename
        # extensions, but in fact the entire path is used. For help with the regexs,
        # see a good book on Perl such as one by Larry Wall.
        
        # Look for common file extensions. 
        handler .*\.(HDF|hdf)(.Z|.gz)*$ hdf
        #handler .*\.(NC|nc|cdf|CDF)$ nc
        handler .*\.(NC|nc|cdf|CDF)$ dap_nc_handler
        handler .*\.(mat|Mat|MAT)$ mat
        handler .*\.(dat|bin)$ ff
        handler .*\.(pvu)(.Z|.gz)*$ dsp
        
        # For JGOFS datasets, match either the dataset name or the absence of an
        # extension. The later case is sort of risky, but if you have lots of JGOFS
        # datasets might be appealing.
        # handler .*/test$ jg
        # handler .*/[^/]+$ jg
        
        
        Wed Jan 04 16:20:01

• This is the dods.rc (symlinked to opendap.rc) we're currently using:

  esmf04m-root> cat dods.rc 
  # -*- Perl -*-
  # OPeNDAP server configuration file.
  # Blank lines are ignored; comments must start with `#' in the first column.
  
  # Set the server time out. This is different from the httpd time out (which
  # does not apply to an OPeNDAP server). The value is in seconds; a value of
  # zero disables the time out feature.
  timeout 0
  
  # Compressed files are decompressed first before serving. Where should these
  # decompressed files be cached?
  cache_dir /usr/tmp
  
  # How big a cache? If you're serving some large files, make sure this is at
  # least big enough to hold one of them! The size is given in MegaBytes.
  cache_size 500
  
  # Email address of the person in charge of this server. 
  maintainer hjm@tacgi.com
  
  # List of handlers whose regular expressions should not be used when building
  # the HTML interface to this server. This includes any overly inclusive
  # regular expression like '.*' which matches anything. This will make README
  # files, et.c, show up as datasets. Typically only the JGOFS server every has
  # these regular expressions. Each handler name should be quoted, separated by
  # whitespace (not a comma).
  #exclude jg
  
  # Match various handlers with pathnames. In *most* cases this means filename
  # extensions, but in fact the entire path is used. For help with the regexs,
  # see a good book on Perl such as one by Larry Wall.
  
  # Look for common file extensions. 
  handler .*\.(HDF|hdf)(.Z|.gz)*$ hdf
  #handler .*\.(NC|nc|cdf|CDF)$ nc
  handler .*\.(NC|nc|cdf|CDF)$ dap_nc_handler
  handler .*\.(mat|Mat|MAT)$ mat
  handler .*\.(dat|bin)$ ff
  handler .*\.(pvu)(.Z|.gz)*$ dsp
  
  # For JGOFS datasets, match either the dataset name or the absence of an
  # extension. The later case is sort of risky, but if you have lots of JGOFS
  # datasets might be appealing.
  # handler .*/test$ jg
  # handler .*/[^/]+$ jg
  

• Based on feedback from a coworker who is familiar with DODS/opendap, these URL's are working fine:
  1. http://esmf.ess.uci.edu/cgi-bin/dods/nph-dods/dodsdata/in.nc.html
  2. http://esmf.ess.uci.edu/cgi-bin/dods/nph-dods/dodsdata/in.nc.das
• But this one is glitching, and a detailed truss doesn't seem to show much about why:
  • http://esmf.ess.uci.edu/cgi-bin/dods/nph-dods/dodsdata/in.nc.dds
  • Output from this URL in the browser:

    Error {
        code = 1002;
        message = "An internal error was encountered:
    Call to set_parent with incorrect variable type.
    Please report this to support@unidata.ucar.edu
    ";
    };
    

• patches to the source
• Configuring (/usr/local/lib/libmalloc.a is a library containing GNU malloc):
  • CC="xlc_r -q64 /usr/local/lib/libmalloc.a"
  • CXX="xlC_r -qlanglvl=extended -q64 /usr/local/lib/libmalloc.a"
  • OBJECT_MODE=64
  • export CC CXX OBJECT_MODE
  • ./configure

• patches to the source
• Configuring:
  • CC="xlc_r -I/usr/local/include/libdap -I. -I./lnetcdf -I./fortran -I/usr/local/include -qlanglvl=extended:redefmac -qcpluscmt -D__inline="
  • CXX="xlC_r -I/usr/local/include/libdap -I. -I./lnetcdf -I./fortran -I/usr/local/include -qlanglvl=extended:redefmac -D__inline="
  • F77="xlf_r -I/usr/local/include/libdap -I. -I./lnetcdf -I./fortran -I/usr/local/include -qlanglvl=extended:redefmac -D__inline="
  • OBJECT_MODE=64
  • LDFLAGS="-L/usr/local/lib -ldap -lC_r"
  • export CC CXX OBJECT_MODE PATH LDFLAGS

You can e-mail the author with questions or comments: