Boost::date_time has changed to throwing a std::out_of_range instead of a
boost::date_time::gregorian::bad_year when a date is outside of the
1400-9999 year range it can deal with.
We also recently discovered that it will use the system localtime function
when creating a new date which can lead to a 2038 failure. Use std::chrono
to resolve that problem.
1. Add dateformat "Locale" with ICU; uses current locale for date
parsing. ICU's locale date parser may parse "3 May 2023" or
"2024年9月13日" (LC_TIME=zh_TW.utf8) and maybe others.
2. Augment d-m-y m-d-y and y-m-d with boost UK/US/ISO parsers. This allows
CSV import of dates with months as words as "30 Sep 2023" or
"May 4, 1978" or "2023-Dec-25". Note boost parser cannot recognise
2-digit years, therefore "30 Sep 24" is invalid.
Extract functions LDT_from_date_time and LDT_from_date_daypart
to avoid duplicate code. Handle date-times in start-of-DST transitions
and better handle those in end-of-DST transitions. Test the results.
The previous fix didn't calculate the comparison time correctly and
ended up in the wrong day because the GMT time is before the local time.
In the course of checking the corner-case timezones (Midway and
Kiritimati) I found an error in the GncDateTime calculation of the
neutral time, so fixed that too.
Revert using boost::locale to generate std::locales as boost::locale-
generated locales don't implement std::locale::facet and there was
a bug in the boost::locale ICU wrapper code that caused the wrong year
to be output for the last 3 days of December.
GCC's libstdc++ supports only the "C" locale on Windows and throws if
one attempts to create any other kind. For dates we work around this
by using wstrftime() to format according to locale and then convert
the UTF16 string to UTF8. wstrftime() interprets the time zone flags
%z, %Z, and %ZP differently so we process those first before calling
strftime. This will have the unfortunate effect of not localizing
timezone names but it's as close as we can get.
It seems that std::locales created by boost::locale::generator are
not entirely compatible: If used to create a new locale with a facet
for boost::date_time one ends up with the C locale and the facet.
For the time being avoid the problem by using boost::locale to format
dates and times. std::chrono gets calendar functions in C++20 so we
can switch date-time backends once we can adopt it.
We can't use std::locale::global because all streams imbue it by
default and if it's not 'C' (aka std::locale::classic) then we
must imbue all the streams that we don't want localized, and that's
most of them.
Provides error checking for setting the C++ locale from the environment.
This is necessary both because the environment might have an invalid
locale, which would cause an unhandled exception crash.
On windows std::locale("") can't handle some Microsoft-style locale
strings (e.g. Spanish_Spain) so we use boost::locale's gen("") function
to set the locale--though even that can't handle a Microsoft-style
locale string with an appended charset (e.g. Spanish_Spain.1252) and
that's what glibc's setlocale(LC_ALL, NULL) emits.
Testing notes: Based on the averages of 3 runs, the net
user CPU to save the XML file I use is:
10.2 seconds without this change
7.6 seconds with this change
In my environment the first call to the format routine
in question, the call that sets the cache value, is at
the end of the XML load.
The first fix for this bug handled structs tm with ambiguous times.
This one fixes the GncDate constructor when the time is ambiguous
because it's in the DST-change hour, using the same add 3 hours,
construct the LDT, and subtract the 3 hours from the result.
The string constructor handles only simple-offset HH:MM timezones and so
is immune to the bug.
strptime/strftime supports various modifiers to their parameters.
'E' and 'O': alternate locale-specific formats
(used in default format for Persian, Oriya, Azerbaijani)
'-': padding
(used in default format for Czech)
GnuCash passes dates as integer y/m/d without using locale-specific
formats, so we need to strip out 'E' and 'O' from the format when
scanning dates or determining separators in gnc-date.
None of '-', 'E', or 'O' are supported by boost (and '-' causes
errors), so strip them out from formatters in gnc-datetime as well.
See https://bugzilla.gnome.org/show_bug.cgi?id=795247.
An odd corner case: BST apparently came off of DST at 23:00 26 Oct 2014,
so midnight that day was ambiguous about being DST or not; that causes
the local_date_time constructor to throw in spite of the tm.is_dst element
being 0 (meaning pick standard time).
Instead of just failing in that case, try constructing a local_date_time
three hours later then adjust it back three hours. If *that* doesn't work
then throw a std::invalid argument.
After much thrashing this turned out to be caused by a date string
with a 3-digit year and that caused an unexpected boost::bad_cast
exception from boost::posix_time::time_from_string().
To prevent that and anything like it, pre-parse the string with
regular expressions to classify them and split out the timezone
if there is one. If neither (perhaps eventually none) of the
regexes match throw std::invalid_argument. The C function will
catch this and return 0.
Turns out the stream facet parser is really slow. Since we have a
well-constrained universe of input formats we don't really need that
overhead. This got a 33% improvement in loading a large SQLite database.
The first attempt to fix this, a17bc85a, doesn't work because the
boost::date_time constructor gets enough information in most cases to
generate a date, just not the one we expect. This change looks for '-' in
the fourth position and if it's there assumes iso-extended format, otherwise
it assumes delimiter-less ISO without the 'T', i.e. %Y%m%d%H%M%S.
This will avoid a ninja-build from picking up a config.h generated by the autotools build
(in the root build directory). Picking up the wrong config.h may lead to all kinds of
subtle issues if the autotools run was done with different options than the cmake run.
It is split into
- /libgnucash (for the non-gui bits)
- /gnucash (for the gui)
- /common (misc source files used by both)
- /bindings (currently only holds python bindings)
This is the first step in restructuring the code. It will need much
more fine tuning later on.
John Ralls
Christopher Lam
Richard Cohen
Christopher D. Carson
YOSHINO Yoshihito
Bill Nottingham
Geert Janssens