Various Software Projects

Most of this, if not all, is used in production. However much of this has not been publicly released.

Some git repos are bare. The repos can be cloned with, e.g.: git clone https://karlpinc.com/software/REPONAME. Repository names are the project name in lower case.

Apologies for the lack of an on-line browser.

I will have to review NDAs before releasing other code and documentation.

Babase (a git repo)

The Babase baboon data management system is designed to facilitate the retrieval, storage, and maintenance of the Amboseli Baboon Project data. The Amboseli Baboon Project has been continuously monitoring a wild baboon population in Kenya's Amboseli National Park since 1971.

It includes an interesting sub-system for PostgreSQL which flags problematic data for later review/cleanup.

Gombe-MI (a git repo)

The Gombe Mother-Infant follows database records and facilitates research done by the Jane Goodall institute's Mother-Infant follow project. Mothers and infants are followed and various aspects of their behavior recorded at regular intervals, including distance to other chimpanzees.

Among other things, it includes the prototype of an interesting regression testing system for PostgreSQL databases.

getcfg (a directory)

A command line tool for use with sh or bash to retrieve a configuration value from a YAML to JSON file.

At present the only documentation is the getcfg --help output.

The linked directory is not a git repo, it contains code yanked from a larger project. I've not yet gotten around to packaging this as its on project.

help.mk (a file)

A Makefile include allowing makefiles to contain embedded documentation which is then displayed to the user. Embedding the documentation of each target in the Makefile's code near (or on the line of) the target allows both the developer, when coding, and the user, when using, easy access to the documentation and avoids having to have separate, duplicated, documentation.

After adding include help.mk to a Makefile, make help displays all lines in the Makefile which begin with ##.

PGWUI (a directory)

A Python framework for interacting with PostgreSQL and modular PGWUI (PostgreSQL Web User Interface) components built with the framework.

At this point there are not a lot of useful components, just the upload bulk_upload and copy components have utility.

The core of PGWUI was one of the first Python program's I'd written. As such, the framework does not have a stable or documented API and so is not ready for release as such. (Particularly the wtforms integration.)

The distinguishing feature of the upload programs is the reporting of multiple errors per batch of uploaded data, while at the same time uploading each batch of data in a transaction so that either all the data in the batch goes into the database or of the data none does. (Note that the upload core is susceptible to SQL injection. This is not a problem in the environments where it is deployed, but it is unsuitable for general release.)

This code was initially developed for the Gombe-MI project. Parts are also included in Babase. The code available here has undergone significant further development.

Component Descriptions and links to not-bare git repos:

common
Files needed by Pyramid apps
core
Core functionality
logout
Logout Page
menu
Menu
server
Web server WSGI functionality
develop
Component development and testing
upload_core
Core files supporting upload
upload
Upload a single file into a table/view
bulk_upload
Upload multiple files into multiple tables/views
copy
Copy a Schema Between Databases
BRIEF-Script (a git repo)

A command line Python program which retrieves a Google Docs spreadsheet, after OAUTH authentication supporting automated access, interpreting the content as a set of constraints on file system directory structure and directory and file naming to ensure that documents regarding lending between clients and banks are properly filed and recorded, including provisions for file naming conventions including proper placement of dashes, parenthesis, brackets, dates, bulleted comments, digits, version numbers, etc. Directory levels in the target file system are represented by column placement in the spreadsheet and name specifications are a custom syntax of text in each cell. A report is produced detailing where the target file system does not meet the specifications of the spreadsheet.

Primate Life History Database Version 2 (web page)

(Note that the plhdb.org web page is extremely dated. Be sure to access version 2 of the source code.)

Migrated database from an Entity-Attribute-Value ontology based database with a custom web interface to a relational model able to utilize any web based PostgreSQL front-end, with the goal being minimal ongoing maintenance. Constructed views and triggers to support data upload and download and incorporated row-level security to control access to the different data sets. Wrote minimal documentation.

This database contains individual-based life history data that have been collected from wild primate populations by nine working group participants over a minimum of 19 years. The purpose of collecting data of this type is to make comparative analyses that can shed light on the population dynamics and the social and ecological adaptations that have shaped both human and nonhuman primate evolution. Records in the database include mortality and fertility schedules across multiple primate taxa. The data are searchable and can be downloaded in CSV format.