Grant Software Innovations - Summary

This is a simple list of the innovations that I have done in a three year period for a client.  All of them build upon my previous innovations, experience, and work at other companies and also on my own time.

[Note to Self- I need to format this better and add some screen shots...]

1.       ETL:

a.       First Generation ETL Program – Command line program in C# to retrieve data from Cosmos DB NoSQL databases, transform the data to a relational form, and insert it into SQL Server for cubing and reporting.

i. Main Points:

1.       Simple and effective ETL via a command line program.

b.       Second Generation ETL Program – Command line program in C# to create a Universal ETL Program for NoSQL to relational data transfer and ETL.

i. Main Points:

1.       Auto adapting ETL program.

2.       Uses Definition Programming for operations instead of endless code changes to adapt to various ETL scenarios and operations.

3.       Can be expanded into a full universal ETL program that can replace a large number of SQL and SSIS ETL implementations.

4.       Fast Evolution (a Super Agile principle/practice) over the first-generation program.

c.       Logging– Created a SQL logging system using a single simple and adaptable logging stored procedure to easily and thoroughly log ETL and proc execution and run time elapsed time, statistics, performance, and automatic error logging.

2.       Tabular Cubes – MCR Cube:

a.       Cosmos DB Data Source - Created first tabular cube to derive its data from a Cosmos DB NoSQL database.

b.       Journaling Confirmation System (JCS) - Created first tabular cube with a secondary data stream to provide auditing confirmation of data processing integrity.

c.       Cube Portal – Built a prototype cube status web portal in ASP.NET MVC that displays the online and last processed date time of reporting cubes as well as the ETL logging information and status. Using SQL logging system I developed.

d.       Cube Deployment Procedures – Developed tabular cube deployment procedures and documentation.

e.       Improved Data Dimension Generation SQL – Fixed and improved that existing date dimension generation SQL to provide improved and expanded date options and coverage.

f. Unified Disputes Cube – Built a prototype cube to unify all the reporting information on all disputes types within Cognosante.

g.       The Cube Answer Book – Developed and wrote The Cube Answer Book the first book to explain OLAP cubes to users and managers in an easy to understand form and to communicate the advantages and uses of cubes, especially tabular OLAP cubes.

h.       Tabular Cube Best Practices – Developed tabular cube best practices.  Awaiting time to finish putting all of them into a book.

3.       Testing:

a.       Test Plans – Developed test plan documents for testing MCR cube and ETL.

b.       Test Documents – Developed test documents for documenting testing of MCR cube and ETL.

c.       Cube Test Tool – Built prototype cube test tool to validate the design and operational implementation approach.

4.       Documentation:

a.       Documentation as You Go – Generated complete ETL and cube documentation as I built the MCR cube and ETL systems for:

i. ETL Design.

ii. ETL Implementation.

iii. Cube Design.

iv. Cube Implementation.

v. Test Plans and Testing Documentation.

vi. Note: This is a Super Agile principle/practice.

5.       Innovative Ideas:

a.       Super Agile – A new next generation software development methodology for super-fast and efficient software design, development, and implementation.  Extends and replaces current limited conventional agile methodology and processes. Awaiting personal time for building website and doing the first short and to the point book.

b.       Smart Components – Why continually re-code the same software functionality over and over again?  Instead build and use Smart Components which encapsulate common software functionality into reusable auto adapting software component modules that replace most common custom software development for database and reporting systems.  Awaiting personal time for building initial demonstration components and doing the first short book.

c.       Definition Programming – Used by Smart Components, Definition Programming is a new way of simply and quickly programming by definition rather than programming by writing a lot of custom procedural code. I have a short PowerPoint presentation on this topic.  And am awaiting personal time to write a short book on this.

6.       Tools and Utilities:

a.       SmartyStreets Web Services SSIS Module – Developed the first SSIS module to interface with SmartyStreets web services from SSIS for MAU (Multiple Address Update) project.  Mentored developer who took over this project.

b.       SmartyStreets Test Tool – A simple and highly useful GUI tool to test both SmartyStreets web service access and check addresses via SmartyStreets web service.

c.       Test Record Generator – Generates thousands of test records from a simple GUI interface.

d.       MCR JSON Test Record Generator – Generates CERRS Connect compatible JSON based test records to a set of test files. Generates one to thousands.

e.       MCR JSON Test Record Poster – Posts one to thousands of the generated JSON test files from the MCR JSON Test Record Generator to the CERRS Connect web service.

f. MCR JSON Test Record Reader – Reads the McrJournalHistory Cosmos DB BSON records posted by CERRS Connect.  Can be used to find, browse, and filter the records to confirm records present and for testing purposes.

g.       Cosmos DB Tool – Enhanced version of the MCR JSON Test Record Reader that allows Cosmos DB browse, finding, and filtering on DevTest, IMPL, and PROD environments for testing purposes.