Oracle RDS Log Files Tools for AWS
This I'm just finishing building...So no screen shot yet. But I'll put one up in the future. Remind me if I'm remiss in this.
So creating and using databases in the Cloud is very easy and convenient. You just spin up a new instance and you're all set in a just a few minutes.
AWS has a set of databases - SQL Server, MySQL, PostgreSQL, and Oracle - As managed database so you can create them in your AWS Cloud quickly and easily. And you don't have to manage them much either.
But with great powers comes great responsibility (where have I heard that before?) as well as sometimes significant limitations. Oracle, and other databases routinely emit and produce multiple files that contain status information for logins (audit log files), trace log files, and error type log files. In a normal on server Oracle installation you just go to the server file directory and you can see and view all of them as they occur...Or the older previous ones. Tail one... Or grep through all of them to find out what the ORA- error code (was it another ORA-0600 again?) is contained in all these files.
And there can be a LOT of files! An ORA-0600 is an internal Oracle server problem and it can generate 1,000 or more files in less than an hour!
Ok, no problem, right! You just do a regular timed grep script to look for ORA- errors in the log files and you can easily see when you get one, where it is, what the other associated log and audit files are...And then you can easily trouble shoot it.
Right?
Ah, no.
Not on AWS Oracle RDS.
You don't have access to the file system...At all.
You can see the log files...But you have to go through the AWS Console and you can only see a list of up to 25 of them at a time. You have to page through them... There is a basic search function that you can use to winnow them down, but when you're looking for 1 or 2 error files amongst a 1,000 files...When it can take days to find the needle in the haystack. Additionally, you have an option to view or tail (look at the more recent entries to a log file in real time)...BUT...Only ONE at a time...And it takes many SECONDS to load...and then up to a minute to go back, select another one, and then load that one.
And...You get a very small and basic windowed area to see the file contents...It's GREEN on the background with black text...VERY low contrast! And you can only see 25 lines or so... And you're limited to the first 1,000 lines in the files. Trace files can be thousands of lines long and up to 1 GB or more in size! This is a really limited and crippled and horrible interface! No grepping or scanning the log files. It's extremely laborious to use this interface.
Ah! But you can DOWNLOAD the files and look at them on your very own PC!
BUT! You can only download ONE FILE AT A TIME...And it's very slow as well!
So after doing putting up with this for several days, looking for the ORA-0600 needle in the 1,000 files haystack, I resolved to build my own tool to access the log files and download them to my local PC where I can easily and quickly grep and view them.
The AWS CLI was a bit limited for this use...And Python had some problems with their AWS SDK being broken with not actually doing all the downloads properly...So over a couple of nights I built my own Oracle RDS Log Files Tool in C# winforms (Windows Desktop App).
It works very well! In the time it took me to build this...Is actually less than the time I would have/did spend trying to go through the AWS Console manually.
This is a tool that I think others in the same situation could really use!