Bug Fixes:

• [1250051] When parsing very large datasets (billions of lines) with multiple processors, an error can occur during log parsing, "Invalid format of PARSED response from parsing server."
• [1256945] Sawmill 8.1 profiles which include "session entrances" or "session exits" as report element columns do not convert properly; the resulting profile will give an error like, '"The report field "ssession_entrances" does not exist in columns of report element "year"' in the Report Editor.
• [1257518] On Windows, if a LogAnalysisInfo has been moved using LogAnalysisInfoDirLoc, the upgrade installer installs the latest version in the default location, rather than in the version specified by LogAnalysisInfoDirLoc.
• [1257840] If there is only one row in the database (typically, because there is only one line in the log data), viewing reports after a rebuild will give an error, "#### Unable to read file LogAnalysisInfo/Databases/apacheextended/main/Tables/xref0/header.cfg (Operation timed out)"
• [1258384] When mapping a drive or share through the Network Disks section of the File Browser, the disk does not immediately appear after being mapped.
• [1258619] After about 2 billion lines processed, the progress display shows the number of line processed as a negative number
• [1258808] If a report field is added by a snapon at profile creation time, and its display format is not "integer," it is overridden by profile creation to be an "integer" display field.
• [1258809] On Windows, if a custom report field displays as "integer," and contains a value larger than about 2 billion, it will display as about 2 billion in HTML reports.
• [1259019] If a syslog-required field has a database field called "duration," the Log Fields Editor shows a blank field label.
• [1259034] After a Remove Database Data operation, the Calendar report still shows links for days that have been removed.
• [1259120] The "concurrent connections" snapon (used to do concurrent connection analysis for several media server plug-ins, including Microsoft Media Server, can overcount concurrent connections in some cases, as it fails to register the end of certain connections. These errors tend to accumulate as more lines are processed, so the number of concurrent connections will incorrectly slow a gradual upward slope.
• [1259396] Fixed a bug where, if using a SQL table prefix or suffix, a Date Range filter would fail with an error like, "SQL query failed: 'select * from bottomleveldateitemnum' error=Table '.bottomleveldateitemnum' doesn't exist"
• [1260066] If the "Log field separator" is a tab, the Config -> Log Processing -> Format page will show an error if you make a change to it, "No value. Please define a value."
• [1260232] Sending mail to a qpsmtpd server, using username/password authentication, gives an error, "503 AUTH not defined for HELO."
• [1260425] When a version 8.1 profile has a "session begin" or "session end" column in a report, and it is converted to 8.5 format, the Report Editor gives an error like, 'The report field "ssession_begin" does not exist in columns of report element "year".'
• [1260484] Viewing Log Detail with a filter which discards all events gives an error like, "Attempt to read beyond end of LogAnalysisInfo/Databases/{profile}/main/Tables/rep_9a7ada7d8bbfe1830f4751a689e58504/data.tbl (fileSize=0); attempted to read from 0 to 280"
• [1260610] Fixed a bug where the Log4j parser did not properly handle %d dates without a curly-bracket format, in PatternLayout (and rejected all entries).
• [1261320] When using an Oracle database, the Log Detail report gives an error like, "Unable to Execute ODBC Query='select x.broken_link, x.date_time, x.day_of_week, x.hour_of_day, x.s_ip, x.cs_method, x.cs_uri_stem, x.cs_uri_stem, x.file_type, x.screen_dimensions, x.screen_depth, x.worm, x.s_port, x.cs_username, x.c_ip, x.domain_description, x.location, x.organization, x.domain, x.isp, x.web_browser, x.operating_system, x.spider, x.sc_status, x.sc_substatus, x.sc_win32_status, x.hits, x.page_views, x.time_taken, x.time_taken, x.session_entrances, x.session_duration, x.bounces from main_table xrownum between 1 AND 50'; diagnostics=ODBC error: rec1: SQLstate: S1000; msg=[Oracle][ODBC][Ora]ORA-00933: SQL command not properly ended ;"
• [1261498] Flash Media Server profiles do not report concurrent connections.
• [1262050] When using an FTP or SFTP log source on Windows, if the log file or pathname contains a colon (:), it gives an error like, "Unable to create folder LogAnalysisInfoTempLogs13301298267768directorysubdir:with:colon"
• [1262299] When using "-a pv" to display database fields summaries from the command line, max/min fields based on date_time display values from 1970.
• [1262381] Temporary tables with names like xref0_update are not properly delete when the database is built, making the database about 25% larger than it needs to be.
• [1262595] If a relative date filter is used which selects just one day, on a Days report with a graph, it will give an error, "#### Unknown variable 'lang_stats.months_short.e_t' in expression". If it is used on a Months report with a graph, it will give an error, "#### Couldn't find node 0 in volatile.temp_month"
• [1262794] Enhanced (and in some cases fixed) memory management, to better limit memory usage. The previous version generally kept its memory usage under control (under the specified limit), but certain types of memory usage were not counted toward the limit, which could result in substantially higher memory usage than allowed.
• [1262847] The Create Profile Wizard does not prompt for the Access Log Valve pattern, when creating a profile using the format, "Tomcat (using Access Log Valve pattern)"
• [1263906] If LogAnalysisInfo is relocated using LogAnalysisInfoDirLoc on Windows, the service will not properly shut down Sawmill.exe processes when it stops.
• [1264170] Create a new report element "Hour of day" with "Display: Graphs". On tab Graph Options change the graph type to "Line graphs". On tab Graphs change sort by to "Hour of day", keep sort direction "Ascending". Click save changes and view the report. The report will show the graph with sort_direction "descending" although the report element editor indicates "ascending".
• [1264241] When using MySQL or MSSQL as the back-end database server, if the database already exists and is already a Sawmill database, and if the profile uses the sessions snapon, an error will occur on profile creation, "Duplicate column name 'session_id'".

• [1102604] Added support for arbitrary (almost) Log4J parsing, through support for most PatternLayout values.
• [1256999] Improved reporting performance has been improved for large reports with "omit parenthesized items" turned on (an example of this is a standard web server "search engines" report). In one example (200 million line dataset), reporting performance increased 24x.
• [1258492] Added support for Clavister SG log format.
• [1258816] Added support for a variant of GroupWise Post Office Agent Log Format, which logs Net Id.
• [1259024] Added a new snapon, "Top Level Domain" which creates a "top level domain" field, and populates it with a log filter, using the list of known top- and -second-level domains to convert a URL to a reasonable top domain name, e.g., "abc.xyz.com" becomes "xyz.com" and "abc.xyz.co.de" becomes "xyz.co.de".
• [1259026] Added a new snapon, "Gateway Reports" which creates a category of four simple reports for gateway decides, for HR purposes: Users Summary, Categories Summary, Domains summary (using the new Domains field snapon), and Usage Detail. The Summary reports include pie charts, and the Usage Detail reports shows category, user, site, start and end time, and duration. This this snapon is attached by default to Palo Alto Integrated, and Squid (without category); other formats will follow.
• [1259220] Greatly improved performance of report generation, for reports containing fields using custom expressions, including "average" fields (e.g., page views per session). Reports using this kind of field, and containing millions of rows, generate as much as 100x faster now. (Reports with few rows are not much affected).
• [1259630] Improved performance of database builds of the internal database, especially multiprocessor builds and profiles without database filters (e.g., profiles without session analysis, or other snapon functionality which creates a database filter). Performance improvements vary by profile, but may be 40%-100% faster than previously, on a multiprocessor system.
• [1259704] Added back support for the rebuild_cross_reference_tables action, which allows all xrefs, or any single xref (with -crt N) to be rebuilt from the command line, without rebuilding all the rest of database filters, or indices.
• [1259821] Added support for numerical reporting of the content_bytes (%B) field, in Apache Custom format strings.
• [1259842] Improved cleanup of Sawmill's "recycling bin" (LogAnalysisInfoTemporaryFilesDeleteMe), by immediately deleting everything put in it (simultaneously with whatever else is going on), rather than waiting for the next cleanup cycle. This can make a huge different in the amount of disk space temporarily used during certain operations, especially 8.1 profile conversion.
• [1260326] Add a new clean_up_database action (e.g. sawmill -p {profile} -a cud), which drops all temporary tables from the database (except those whose parent process is still running). This also now occurs automatically at the beginning of any database update, or "remove database data." This eliminates the clutter which sometimes results when reports or other processes terminate abnormally, and fail to clean up their temporary tables.
• [1260480] Added functionality to include multiple log filter initializations, and multiple log filter finalizations, in a single profile (as subnodes of log.filter_initialization and log.filter_finalization). They are run in order. Old-style initializations and finalizations (expressions directly on log.filter_initialization, etc.) are still supported. Added a new snapon operation to add a filter initialization or finalization to a profile. Together, these features allow snapons to add independent filter initializations and finalizations to a profile.
• [1260482] Implemented the "Advanced Example: Rejecting spiders based on JS and /robots.txt access" log filter example as a snapon, for much easier implementation in a profile.
• [1261082] Added a new action, rebuild_database_filters, which rebuilds all database filters.
• [1262382] Enhanced Wowza analysis to use the new concurrency snapon to track concurrent streams, instead of the older "session" style analysis.
• [1264537] Added support for Microsoft Forefront log format.

Bug Fixes:

• [1250051] When parsing very large datasets (billions of lines) with multiple processors, an error can occur during log parsing, "Invalid format of PARSED response from parsing server."
• [1256945] Sawmill 8.1 profiles which include "session entrances" or "session exits" as report element columns do not convert properly; the resulting profile will more...