Merge pull request #5 from lisalenorelowe/master

Instructions for Ubuntu and Mac

README.md

@@ -0,0 +1,12 @@
+# schism_visit_plugin
+VisIT plugins for visualization of VisIt gr3, prop and output files
+
+Install instructions tested with VisIt 3.1.4
+- [install-macm1-visit3.1.4](install-macm1-visit3.1.4.md) - Installing on a Mac with an M1 chip but targeting x86_64 architecture
+- [install-ubuntu20-visit3.1.4](install-ubuntu20-visit3.1.4.md) - Installing on Ubuntu 20
+
+Get sample inputs here: [SCHISM Visualization](https://schism-dev.github.io/schism/master/getting-started/visualization.html#visualization-with-visit)
+
+Install instructions have `wget` commands for downloading these from the command line.
+
+When using the new output format (with scribes), be sure to select "SCHISM" when reading in the file, i.e., File:Open file:Guess from file name/extension:SCHISM.

environment.yml

@@ -0,0 +1,36 @@
+name: pyschism
+channels:
+  - conda-forge
+dependencies:
+  - f90nml
+  - numpy 
+  - gfortran
+  - pandas
+  - matplotlib
+  - pygeos
+  - geopandas
+  - netcdf-flattener>=1.2.0
+  - netCDF4
+  - psutil
+  - scipy
+  - wget
+  - cf-python
+  - metpy
+  - sqlalchemy
+  - pyugrid
+  - boto3
+  - rtree
+  - numba
+  - tqdm
+  - xmltodict
+  - cdsapi
+  - seawater
+  - xarray
+  - cfgrib
+  - zarr
+  - fsspec
+  - stormevents
+  - utm
+  - pyproj
+  - pyshp
+  - appdirs

install-expanse.md

@@ -0,0 +1,156 @@
+You can install the plugin in your home directory.
+
+# Set the VisIt environment
+
+Currently, there is no 'visit' module, and I think that is because you should only use it in client-server mode.  But we need the xml stuff to compile the plugins.
+
+The instructions for running in client-server mode is here: `/cm/shared/examples/sdsc/visit/README`.
+
+From that, you can get the path: `/cm/shared/apps/vis/visit/3.1.4/gcc/9.2.0/openmpi/3.1.6/3.1.4/linux-x86_64/bin`, which indicates that VisIt 3.1.4 was compiled with gcc 9.2.0 and openmpi 3.1.6.
+
+First, load those gcc and openmpi modules, and load cmake.
+```
+module load gcc/9.2.0
+module load openmpi/3.1.6
+module load cmake
+```
+
+Here is my modulefile for VisIt 3.1.4.
+```
+#%Module
+set VISITARCHHOME "/cm/shared/apps/vis/visit/3.1.4/gcc/9.2.0/openmpi/3.1.6/3.1.4/linux-x86_64"
+setenv VISITARCHHOME "/cm/shared/apps/vis/visit/3.1.4/gcc/9.2.0/openmpi/3.1.6/3.1.4/linux-x86_64"
+prepend-path PATH "$VISITARCHHOME/bin"
+prepend-path LD_LIBRARY_PATH "$VISITARCHHOME/lib"
+setenv VISITPLUGININSTPRI "/home/llowe/.visit/3.1.4/linux-x86_64/plugins/"
+```
+
+And I have this in my .bashrc:
+```
+module use --append /home/llowe/modulefiles
+```
+
+So now load visit
+```
+module load visit
+```
+
+If you don't want to make a module, do:
+```
+export PATH=/cm/shared/apps/vis/visit/3.1.4/gcc/9.2.0/openmpi/3.1.6/3.1.4/linux-x86_64/bin:$PATH
+```
+
+# Install the plugin
+
+Now try the steps to install the plugin.
+
+Get the plugin code.  This one test *my* plugin code with the malloc.h in ifdef statements:
+```
+git clone https://github.com/lisalenorelowe/schism_visit_plugin.git
+cd schism_visit_plugin
+```
+
+Install the unstructure_data plugin:
+```
+cd unstructure_data
+```
+
+The commands with `xml2` are VisIt commands.
+```
+xml2cmake -clobber SCHISMOutput.xml
+xml2info SCHISMOutput.xml
+```
+SCHISMOutput.xml is the file used by VisIt code generating tool to create code skeleton and makelist file.
+
+Make a build directory:
+```
+mkdir build
+cd build
+```
+
+Use `cmake` to create the `make` system.
+```
+cmake -DCMAKE_BUILD_TYPE:STRING=Release -S .. -B .
+```
+
+Run `make` to build plugins binary. 
+```
+make
+```
+Check:
+```
+ls ~/.visit/3.1.4/linux-x86_64/plugins/databases
+```
+There should be four new files in ~/.visit/3.1.4/linux-x86_64/plugins/databases:
+```
+libESCHISMDatabase_par.so
+libESCHISMDatabase_ser.so
+libISCHISMDatabase_par.so
+libMSCHISMDatabase_par.so
+```
+
+Repeat the steps for the other plugins:
+```
+cd ~/schism_visit_plugin/prop
+xml2cmake -clobber prop.xml
+xml2info prop.xml
+mkdir build
+cd build
+cmake -DCMAKE_BUILD_TYPE:STRING=Release -S .. -B .
+make
+ls ~/.visit/3.1.4/linux-x86_64/plugins/databases
+```
+
+And
+```
+cd ~/schism_visit_plugin/gr3
+xml2cmake -clobber gr3.xml
+xml2info gr3.xml
+mkdir build
+cd build
+cmake -DCMAKE_BUILD_TYPE:STRING=Release ..
+make
+ls ~/.visit/3.1.4/linux-x86_64/plugins/databases
+```
+Works.
+
+And
+```
+cd ~/schism_visit_plugin/mdschism
+xml2cmake -clobber mdschism.xml
+xml2info mdschism.xml
+mkdir build
+cd build
+cmake -DCMAKE_BUILD_TYPE:STRING=Release ..
+make
+```
+Error.
+
+So, this is where we hack the makefiles, as per Eric's email (see below), 
+```
+vi CMakeFiles/EMDSCHISMDatabase_par.dir/flags.make
+vi CMakeFiles/MMDSCHISMDatabase.dir/flags.make
+vi CMakeFiles/EMDSCHISMDatabase_ser.dir/flags.make
+vi CMakeFiles/IMDSCHISMDatabase.dir/flags.make
+```
+and remove all the
+```
+libnetcdf_c++.a libnetcdf.a libhdf5_hl.so libhdf5.so libsz.so libz.so
+```
+and then, do make:
+```
+make
+```
+Check that you have all the plugins now:
+```
+ls ~/.visit/3.1.4/linux-x86_64/plugins/databases
+```
+You should have the following.
+```
+libEMDSCHISMDatabase_par.so  libEpropDatabase_par.so  libMMDSCHISMDatabase.so
+libEMDSCHISMDatabase_ser.so  libEpropDatabase_ser.so  libMSCHISMDatabase.so
+libESCHISMDatabase_par.so    libIMDSCHISMDatabase.so  libMgr3Database.so
+libESCHISMDatabase_ser.so    libISCHISMDatabase.so    libMpropDatabase.so
+libEgr3Database_par.so	     libIgr3Database.so
+libEgr3Database_ser.so	     libIpropDatabase.so
+```

install-macm1-visit3.1.4.md

@@ -0,0 +1,174 @@
+# Installing on Mac Studio
+
+Installing on my Mac Studio that has an M1 chip.
+
+Using VisIt 3.1.4, to use client-server mode from Expanse, which is only available as x86_64.
+
+Download CMake for Mac, drag it into the applications bin.  Open CMake, in the top menu bar, click Tools, 
+choose How to Install For Command Line Use.  I chose this way:
+```
+sudo "/Applications/CMake.app/Contents/bin/cmake-gui" --install
+```
+
+When compiling, I got a bunch of errors because of `malloc.h`.  Searching on the internets, I found [this possible solution](https://github.com/RIOT-OS/RIOT/issues/2361).
+So for every subroutine where there is `malloc.h`, I did this:
+```
+#if defined(__MACH__)
+#include <stdlib.h>
+#else
+#include <malloc.h>
+#endif
+```
+These were the files:
+```
+(base) lllowe@LisasMacStudio schism_visit_plugin % grep malloc.h */*
+mdschism/avtMDSCHISMFileFormatImpl.C
+unstructure_data/avtSCHISMFileFormat.C
+unstructure_data/avtSCHISMFileFormatImpl.C
+unstructure_data/avtSCHISMFileFormatImpl10.C
+unstructure_data/avtSCHISMFileFormatImpl11.C
+```
+
+On my fork, it is fixed, so get that one:
+```
+git clone https://github.com/lisalenorelowe/schism_visit_plugin.git
+cd schism_visit_plugin
+```
+
+Install the unstructure_data plugin:
+```
+cd unstructure_data
+```
+Save a copy of the main VisIt plugin class files to your home directory. These files will be overwritten in the next step by VisIt's code skeleton generating tool.
+```
+cp avtSCHISMFileFormat.C ~
+cp avtSCHISMFileFormat.h ~
+```
+The command `xml2plugin` is a VisIt command.
+```
+xml2plugin -clobber SCHISMOutput.xml
+```
+SCHISMOutput.xml is the file used by VisIt code generating tool to create code skeleton and makelist file.
+
+Use two backup plugin class file to overwrite the files avtSCHISMFileFormat.C and avtSCHISMFileFormat.h.
+```
+cp ~/avtSCHISMFileFormat.C .
+cp ~/avtSCHISMFileFormat.h .
+```
+
+Make a build directory:
+```
+mkdir build
+cd build
+```
+
+Use `cmake` to create the `make` system.  My mac is arm but the executable for VisIt 3.1.4 is x86_64, so the next command tells it to target x86.
+```
+cmake -DCMAKE_BUILD_TYPE:STRING=Release -DCMAKE_OSX_ARCHITECTURES=x86_64 ..
+```
+
+Run `make` to build plugins binary. 
+```
+make
+```
+
+Check
+```
+ls ~/.visit/3.1.4/darwin-x86_64/plugins/databases
+```
+
+There should be four new files in ~/.visit/3.1.4/darwin-x86_64/plugins/databases:
+```
+libESCHISMDatabase_par.so
+libESCHISMDatabase_ser.so
+libISCHISMDatabase_par.so
+libMSCHISMDatabase_par.so
+```
+
+
+Repeat the steps for the other plugins:
+```
+cd ~/schism_visit_plugin/prop
+cp avtpropFileFormat.C ~
+cp avtpropFileFormat.h ~
+xml2plugin -clobber prop.xml
+cp ~/avtpropFileFormat.C .
+cp ~/avtpropFileFormat.h .
+mkdir build
+cd build
+cmake -DCMAKE_BUILD_TYPE:STRING=Release -DCMAKE_OSX_ARCHITECTURES=x86_64 ..
+make
+ls ~/.visit/3.1.4/darwin-x86_64/plugins/databases
+```
+Works.
+
+And
+```
+cd ~/schism_visit_plugin/gr3
+cp avtgr3FileFormat.C ~
+cp avtgr3FileFormat.h ~
+xml2plugin -clobber gr3.xml
+cp ~/avtgr3FileFormat.C .
+cp ~/avtgr3FileFormat.h .
+mkdir build
+cd build
+cmake -DCMAKE_BUILD_TYPE:STRING=Release -DCMAKE_OSX_ARCHITECTURES=x86_64 ..
+make
+ls ~/.visit/3.1.4/darwin-x86_64/plugins/databases
+```
+Works.
+
+And
+```
+cd ~/schism_visit_plugin/mdschism
+cp avtMDSCHISMFileFormat.C ~
+cp avtMDSCHISMFileFormat.h ~
+xml2plugin -clobber mdschism.xml
+cp ~/avtMDSCHISMFileFormat.C .
+cp ~/avtMDSCHISMFileFormat.h .
+mkdir build
+cd build
+cmake -DCMAKE_BUILD_TYPE:STRING=Release -DCMAKE_OSX_ARCHITECTURES=x86_64 ..
+make
+```
+Error.  
+
+So, this is where we hack the makefiles, as per Eric's email, 
+```
+vi CMakeFiles/EMDSCHISMDatabase_par.dir/flags.make
+vi CMakeFiles/MMDSCHISMDatabase.dir/flags.make
+vi CMakeFiles/EMDSCHISMDatabase_ser.dir/flags.make
+vi CMakeFiles/IMDSCHISMDatabase.dir/flags.make
+```
+remove all the
+```
+libnetcdf_c++.dylib libnetcdf.dylib libhdf5_hl.dylib libhdf5.dylib libsz.dylib libz.dylib
+```
+
+Then make
+```
+make
+```
+
+Check
+```
+ls ~/.visit/3.1.4/darwin-x86_64/plugins/databases
+```
+
+
+And here you go:
+```
+(base) lllowe@LisasMacStudio build % ls ~/.visit/3.1.4/darwin-x86_64/plugins/databases
+libEMDSCHISMDatabase_par.dylib libEgr3Database_par.dylib      libIMDSCHISMDatabase.dylib     libMMDSCHISMDatabase.dylib
+libEMDSCHISMDatabase_ser.dylib libEgr3Database_ser.dylib      libISCHISMDatabase.dylib       libMSCHISMDatabase.dylib
+libESCHISMDatabase_par.dylib   libEpropDatabase_par.dylib     libIgr3Database.dylib          libMgr3Database.dylib
+libESCHISMDatabase_ser.dylib   libEpropDatabase_ser.dylib     libIpropDatabase.dylib         libMpropDatabase.dylib
+```
+
+
+Now try it out.  Links to data are on the [SCHISM visualization page](https://schism-dev.github.io/schism/master/getting-started/visualization.html)
+
+```
+wget -r -nH --cut-dirs=2 -np -R "index.html*" http://ccrm.vims.edu/yinglong/SVN_large_files/Scribe_IO_outputs/
+wget -r -nH --cut-dirs=2 -np -R "index.html*" http://ccrm.vims.edu/yinglong/SVN_large_files/SCHISM_v5.6.1_sample_outputs/
+```