| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
In server mode, when the merging after a 409 from the DB determines
that the client already has the latest data, sending it a Replace
command can be avoided.
This patch achieves that by introducing a new boolean
remoteHasLatestData which is false by default and only set when it is
certain that sending the update can be avoided. To minimize changes,
the DB_DataReplaced code path is left unchanged. It could be changed
to set the same boolean.
Not sure whether that will also avoid unnecessary changes sent when a
client detects a conflict.
|
|
TLocalEngineDS::engProcessRemoteItemAsServer() in localengineds.cpp
counts Add commands as "added items", even if later on a 409 DB result
causes the item to be updated or be left unmodified.
Instead changing the whole statistics logic, this patch merely fixes
the statistics locally in the 409 handling code.
|
|
Instead of always assuming that the incoming item is meant to win,
look at the age of the two items first.
Also tell the code which updates the DB item whether it was changed at
all and skip the DB write if it is not needed. Note that statistics
are still wrong: a server add is counted as "item added" in all cases,
even if it is turned into an update or becomes a nop.
|
|
When the 409 handling call falls back to the older merge handling code,
that code ends up logging 207 as the status. Replaced with a full list
of possible reasons why the code might have been invoked.
|
|
The compare result was only 1 or 0, never -1, because of a teriary
operator which mapped all non-null values to 1. Cut-and-paste error?
The fix passes the underlying value through unchanged. Perhaps this
will return values like -999 to the caller, in contradiction to the
definition of the macro. However, it is unclear what should be
returned in that case.
|
|
When a merged item is added to the items which need to be sent with
SendDBVersionOfItemAsServer(), the same item must not be freed again
immediately because the send queue holds another pointer to it.
|
|
New feature added: DB_DataChanged/Replaced/Conflict support.
|
|
For these codes the local ID must be passed through to the caller.
For DB_Conflict no real changes were made in the plugin, so after
write scripts and blog writing are not invoked.
|
|
Plugins are now allowed to return LOCERR_OK and DB_DataMerged. The
latter must be treated like success at the API level. Special
treatment then happens in the caller.
|
|
compile problem
After the recent change ("server engine: better support for backend
doing its own duplicate merging (status 207 from API)"),
dataconversion.cpp no longer compiled because the dummy engine didn't
implement the new pure virtual getConflictingItemByLocalID().
|
|
Up to now, only DB_DataMerged (207) had a special meaning when
returned by the DB backend for a server add.
Now the following options exist for server add operations:
- DB backend returns DB_DataMerged (207):
This means that the backend has augmented the item
with additional data, either from an external source,
or by merging with another item in the sync set.
The backend must return the localID for the augmented
item. In case the augmented item was created by
combining with a existing item already in the sync set,
the localID of that existing item must be returned
(and not a new ID!)
The client will be updated with the augmented
version.
Additionally, if the new version
was created by merge with an existing item in the
sync set, a delete is sent to the client to
remove the old (pre-merge) version of the item
from the client.
- DB backend returns DB_DataReplaced (208):
This means that the backend has stored the item,
but doing so, an existing item from the sync set
was replaced.
The backend must return the localID for the
replaced item which must also become the localID
of the newly stored version.
The client item will not be updated (as the item
was stored as received), but otherwise the processing
is the same as described for DB_DataMerged (207) above.
Additionally, for both server and client, add and replace,
the following new option exists:
- DB backend returns DB_Conflict (409)
This means that the backend did NOT store the item,
but has detected a need to merge the current version
stored in the backend with the new version.
The engine will read the current version,
merge it with the incoming version, and issue
an updateItem operation on the backend with
the merge result.
In server add case, after the merge the same
processing occurs as for DB_DataMerged (see above).
|
|
(status 207 from API)
So far, the DB backend could return DB_DataMerged (=207) for an add
to signal that the added data was merged with some pre-existing data
and thus the client should be updated with the merge result. This was
intended for merge with external data (augmenting entries with
lookups from other sources), but not yet for duplicate elimination.
This patch adds support for propagating merges that eliminate duplicates
in addition to merges that just add external data.
When a client sends a new item (new = client side ID not yet known
server side), and the DB backend returns status 207 for AddItem,
the following happens:
- current maps are searched for an item with the same localID as
what the DB backend just returned as the "new" localID. If such
an item is found, this means the add was completed by merging
with an existing item.
- If so, this means that for this single localID, there are now
two versions with different remoteIDs on the client. The item
with the remoteID found in the map must be deleted, so a
delete command is added to the list of changes to be sent to
the client.
- It might also be that the item merged was just added to the
server (and not yet known to the client), or had another
change predating the merge. If so, the list of changes to be sent
to the client will contain an add or a replace, resp.
that must NOT propagate to the client, so it is removed
from the list now.
- Finally, the result of the merge is fetched from the database
and propagated to the client as a replace command.
|
|
Switching between distros only works when wiping out the wrong m4
macros. Use the same approach as in SyncEvolution: temporary macros
in m4, permanent ones under git control in m4-repo.
|
|
Use --enable-warnings=fatal to compile with warnings enabled which the
code needs to pass and these warnings turned into errors. Easier than
having to remember and set the right CFLAGS/CXXFLAGS.
|
|
The way the code was before, a sequence like "foo\,bar" would have
been converted into "foo,,bar", i.e. duplicated the escaped char
instead of retaining the escape.
|
|
Copying the "char *" into std::string is more efficient with
std::string::assign() than with the assignment operator because the
already known string length can be reused.
As a side effect it also avoids a valgrind warning about reading one
byte past the end of the allocated string buffer inside char_traits.h
line 261. Probably was cause by an optimized strlen() variant inside
std::string.
|
|
The first four digits are from the upstream tag. The last digit needs
to be incremented each time relevant changes are released as part of
the Linux libsynthesis release in SyncEvolution. This is relevant
primarily for the PKG_CONFIG_CHECK(), which now can check that the
libsynthesis that is installed is recent enough.
|
|
Ubuntu Oneiric
|
|
that should be rendered as B64 only in case they are really non-printable or non-ASCII
This is an addition to 8d5cce896d ("blob" fields: avoid binary encoding if possible) to avoid change of behaviour for CONVMODE_BLOB_B64.
|
|
The READ(filename) method returns the content of the file identified
with "filename". Relative paths are interpreted relative to the current
directory. On failures, an error messages is logged and UNASSIGNED
is returned.
This method is useful for inlining the photo data referenced with
local file:// URIs shortly before sending to a remote peer. SyncEvolution
uses the method in its outgoing vcard script as follows:
Field list:
<!-- Photo -->
<field name="PHOTO" type="blob" compare="never" merge="fillempty"/>
<field name="PHOTO_TYPE" type="string" compare="never" merge="fillempty"/>
<field name="PHOTO_VALUE" type="string" compare="never" merge="fillempty"/>
Profile:
<property name="PHOTO" filter="no">
<value field="PHOTO" conversion="BLOB_B64"/>
<parameter name="TYPE" default="no" show="yes">
<value field="PHOTO_TYPE"/>
</parameter>
<parameter name="VALUE" default="no" show="yes">
<value field="PHOTO_VALUE"/>
</parameter>
</property>
Script:
if (PHOTO_VALUE == "uri" &&
SUBSTR(PHOTO, 0, 7) == "file://") {
// inline the photo data
string data;
data = READ(SUBSTR(PHOTO, 7));
if (data != UNASSIGNED) {
PHOTO = data;
PHOTO_VALUE = "binary";
}
}
Test cases for inlining, not inlining because of non-file URI and
failed inling (file not found) were added to SyncEvolution.
|
|
If the text goes through macro expansion, then "aScriptText" is not
the chunk of memory which holds the script and "text" doesn't point
into it anymore. Therefore "text-aScriptText" calculates the wrong
offset.
Fixed by storing the real start of memory in a different variable
and using that instead of aScriptText.
Found when enclosing a string with single quotes instead of double
quotes. The resulting syntax error message contained garbled
characters instead of the real script line.
|
|
This change is meant for the PHOTO value, which can contain both
binary data and plain text URIs. Other binary data fields might also
benefit when their content turns out to be plain text (shorter
encoding).
The change is that base64 encoding is not enforced if all characters
are ASCII and printable. That allows special characters like colon,
comma, and semicolon to appear unchanged in the content.
Regardless whether the check succeeds, the result is guaranteed to
contain only ASCII characters, either because it only contains those
to start with or because of the base64 encoding.
|
|
g++ 4.6 warns that the "rc" variable is getting assigned but never
used. smlEndEvaluation() must have been meant to return that error
code instead of always returning SML_ERR_OK - fixed.
|
|
The 3529d3cb commit which introduced support for "unprocessed" wildcard
properties added the logic that such properties are always considered
"available" regardless of the actual CtCap of the peer.
This has the undesirable effect that the content of these properties
are removed when receiving an update from a peer which doesn't
support them.
Therefore this commit reverts that part of 3529d3cb: setfieldoptions()
with NULL as first parameter now only enables mandatory properties,
as it did before the introduction of "unprocessed" properties.
"unprocessed" properties need to be listed explicitly in the CtCap
like any other property. This is currently untested.
|
|
|
|
A compare script must know what the compare mode is. When checking for
"items equal", it must return 0 (equal) or -999 (not equal). When doing an
age comparison, it must 1 or -1.
The new COMPAREMODE() method returns a string which identifies the current
mode:
"all", // compare all fields, even irrelevant ones
"n/a", // n/a (scripted)
"conflict", // compare fields relevant for (normal sync) conflict detection
"slowsync", // compare fields relevant for slow sync match
"firstsync", // compare fields relevant for first time slow sync match (possibly relaxed comparison rules)
"age", // no test for equality, only for age (returns SYSYNC_NOT_COMPARABLE if age comparison not possible)
A script only needs to check for "age", and it equal, do the age comparison
instead of checking for equality.
|
|
These functions are used by the XML DevInf parser and thus must
be exported by libsmltk.so, otherwise linking fails.
Perhaps a different API should be used instead. Exporting these
functions seemed like the simpler solution for the moment.
|
|
Now the DevInf is parsed in TSessionConfig::localResolve(). The result
is stored in the TRemoteRuleConfig instance.
The advantage is that syntax errors are found earlier and for all
remote rules, not just the one active during a sync. The downside is
somewhat increased memory and CPU consumption.
The DevInf in XML format is kept in memory although no longer needed
after localResolve(). That might be useful for debugging.
|
|
Many SyncML servers send DevInf without CtCap inside. For those
servers, the engine (often incorrectly) has to assume that they
support all fields.
This patch adds a configuration element "OverrideDevInf" which helps
to address this problem. It may occur in a "RemoteRule". Once that
rule matches a DevInf sent by a peer, processing of CtCap continues
with the CtCap element of the DevInf inside the remote rule. This
allows adding and overriding the CtCap from a peer.
"OverrideDevInf" must contain a complete DevInf, although right now
only the CtCap inside it really matters.
This approach has the advantage that it can reuse the existing and
somewhat documented CtCap format for describing capabilities. The
downside is that not everything can be encoded in that format, for example,
how many components of the ORG field are supported. A possible
solution for that is to extend the format in a way that only the Synthesis
engine understand and/or standardize such extensions.
Right now the "OverrideDevInf" content is only parsed when needed.
Beware that syntax errors will only be found when using the rule!
|
|
Hide the pragmas inside ifdefs to avoid complaints from other
compilers about unknown pragmas.
|
|
The name clash is okay in this case. The two virtual methods
really can be overridden separately. Tell clang that via
pragmas. The pragmas themselves are inside ifdefs to avoid
complaints from other compilers about unknown pragmas.
|
|
This reverts commit 487d78832ca43879902dd0a119fcc2015c50e88c.
Will fix the clang compiler warning differently.
As Lukas explained:
These two versions of apiEndDataWrite() must both exist. It's a historic thing:
- TBinfileImplDS always had no-op apiEndDataWrite(void), that *could* be overridden
by direct descendant (which exist e.g. in non-OS WinMobile and PalmOS datastores)
- TCustomImplDS always had abstract apiEndDataWrite(string &) which has to be
implemented in a descendant (and is, in odbcabidb and pluginapidb).
|
|
gcc 4.6 seems to change header dependencies. cstddef must
be included explicitly. Kudos to Peter Robinson for
reporting this and providing the patch.
|
|
The code added in commit e8b3e088fdc894cade81472412817c01c7cef57f
only acticated sub-rules directly included by the rule specified
as parameter of MAKE/PARSETEXTWITHPROFILE().
With this patch, TMimeDirProfileHandler::activateRemoteRule()
activates sub-rules recursively. As before the settings of
a rule are activated before activating sub-rules, so they
may overwrite their parent's settings again.
Because setRemoteRule/activateRemoteRule() are only used
by scripts, syncing is not affected by this change. I haven't
checked whether recursive inclusion is handled there.
|
|
In a client, PEV_ALERTED was reported to the app with the final sync
mode. On the server, that was missing, partly because the
determination of the final sync mode has to be delayed a bit.
This patch moves the progress event into the one place that is called
in all cases: TLocalEngineDS::changeState() for aNewState ==
dssta_syncmodestable. There it is executed for client and server.
|
|
clang 2.9 detects comparisons which are always true, like "unsigned
value >= 0", and warns about them. In some cases the usage of a signed
type prevents detecting underflows (argidx = c-'1'), in another it
means that error handling was never done (fread() returns 0 on error,
not -1).
Fixed by switching (back?) to signed types and casting to unsigned
after sanity check and before comparison against other, real unsigned
types.
|
|
clang 2.9 pointed out that apiEndDataWrite() existed in versions
with and without parameter. Unified that to the version with
a string retval.
Probably never mattered because all implementations are empty?!
|
|
libsynthesissdk calls libsmltk directly (in san.cpp). libsmltk used to
be provided via libsynthesis, but only if really linked (not always
the case, for example when using the SDK in "open engine dynamically"
mode) and indirect linking stopped working with more recent Linux
bintools which enforce that all libraries are properly declared.
This patch fixes this both in libtool linking (.la files, LIBADD) and
pkg-config definition files (.pc files).
|
|
The VERSION set by AM_INIT_AUTOMAKE() was only used in the *.pc files
and has never been updated since the initial Linux release. Bumped it
to 3.4, so that SyncEvolution can test for a recent enough libsynthesis
during configuration.
Also incremented the .so. New features were added, without breaking
backwards compatibility. This is not quite the same as extending the
ABI, but close.
|
|
With these fixes, clang -Wall -Werror -Wno-unknown-pragmas compiles
cleanly.
Changes:
- short <-> int mismatches in printf formatting
- NULL format string causes a warning (okay in DebugOpenBlock(), but
cannot tell the compiler -> disable checking in that one place
where it happens)
- return NULL pointer instead of implicitly converting false to NULL
- use curly brackets to denote empty body because clang does not
accept a single colon
|
|
|
|
encoded properties not to be decoded any more. This was introduced with 3529d3cb09
- only for unprocessed properties, encodings other than QP must not be decoded.
This condition was missing, so normal B64 encoded PHOTO etc. was no longer
stored as decoded binary in BLOB fields as intended, but stored as-is.
- improved comments regarding "unprocessed"
- some whitespace cosmetics
|
|
Conflicts:
src/sysync/mimedirprofile.h
Some conflict around the added comments.
|
|
modes like BDAY, which is removed again)
In <value>, the "conversion" attribute now can not only
contain the basic conversion mode, but also some
options, which are appended with a + sign:
<value field="BDAY" conversion="DATE+EXTFMT"/>
The defined options so far are:
- EXTFMT: display dates and times in ISO8601 extended format
- MILLISEC: show time with millisecond resolution
|
|
case for this property, even in vCard 3.0).
Notes:
- parsing ISO8601 extended format was implemented for
a while already (for all dates/times)
- in vCard 2.1 both extended and standard format is
allowed. In vCard 3.0 only extended format is in the specs.
|
|
|
|
unknown extensions (like X-xxxx properies)
The properties matched by a wildcard definiton (usually "X-*", see example below)
are stored entirely (including name and parameters) in item fields.
However, the properties are stored not totally unprocessed, but in a normalized form
that removes any dependency from old vCard 2.1/vCalendar 1.0 vs vCard 3.0/iCalendar 2.0
specifics.
This allows reading X-xxx properties e.g. from a vCard 2.1 and reproducing them in a
vCard 3.0 and vice versa.
The normalized form is as follows:
- Folding is removed
- strings are always UTF-8. When parsing a old vXXX-format with a CHARSET parameter indicating
a different charset, the strings are converted to UTF-8 and the CHARSET parameter is
removed from the normalized form.
- Linefeeds within the data (multi-line strings) are represented as plain LF characters within strings
- Quoted-Printable encoding (often needed in old vXXX formats to represent multi-line content) is
decoded and the ENCODING parameter is removed from the normalized form.
- Backslash escaping is NOT removed (except for the special case of \n in new vXXX formats
which is converted to a real LF character).
An example:
- In the config, define a array for the unknown properties
<fieldlist>
...
<field name="XPROPS" array="yes" type="string" compare="never"/>
</fieldlist>
- add a wildcard property definition at the end of the profile (for calendar, insert
a copy into both VTODO and VEVENT subprofiles).
It would also be possible catching all unknown properties using "*", but make
sure the wildcard specs comes after all other properties because the first match
is used when parsing.
Properties with a wildcard name must have a single <value> spec which should
be a string field/array which receives the normalized form of the matched
properties found.
<property name="X-*" suppressempty="yes" show="false">
<value field="XPROPS"/>
<position field="XPROPS" repeat="array" increment="1" minshow="0"/>
</property>
Now a property in a iCalendar 2.0 (without folding here) like:
X-TEST;PARAMETER1=nonquoted;PARAMETER2="quoted and with separators inside:;,":Content with\nMultiple\nText lines\nand national chars: äöü
becomes
X-TEST;PARAMETER1=nonquoted;PARAMETER2="quoted and with separators inside:;,":Content with
Multiple
Text lines
and national chars: äöü
in the internal representation and renders back into iCalendar 2.0 as:
X-TEST;PARAMETER1=nonquoted;PARAMETER2="quoted and with separators inside:;
,":Content with\nMultiple\nText lines\nand national chars: äöü
or as vCalendar 1.0:
X-TEST;PARAMETER1=nonquoted;PARAMETER2="quoted and with separators
inside:;,";CHARSET=UTF-8;ENCODING=QUOTED-PRINTABLE:Content with=0D=0A=
Multiple=0D=0A=
Text lines=0D=0A=
and national chars: =C3=A4=C3=B6=C3=BC
Assuming the property comes from a device that only does ISO-8859-1 and vCalendar 1.0 it might
look like:
X-TEST;PARAMETER1=nonquoted;PARAMETER2="quoted and with separators
inside:;,";CHARSET=ISO-8859-1;ENCODING=QUOTED-PRINTABLE:Content with=0D=0A=
Multiple=0D=0A=
Text lines=0D=0A=
and national chars: =E4=F6=FC
The internal representation will be the same as above (UTF-8 and CHARSET and ENCODING params removed):
X-TEST;PARAMETER1=nonquoted;PARAMETER2="quoted and with separators inside:;,":Content with
Multiple
Text lines
and national chars: äöü
So this allows converting back and forth between old and new vXXX formats for many unknown
extension properties, even if these contain non-trivial data with national chars and linefeeds.
Of course, more complex properties like those containg date/time with TZIDs etc.
will not transfer semantically correct, but still syntactically ok such that a parser should
be able at least to read over them without stumbling.
|
|
conversion options
So far the optional rule parameter was only used to select properties
in the profile via their "rule" parameter. In contrast to a sync session,
it wasn't possible to enable <noemptyproperties>, for example.
This patch makes that possible by extending the
TMimeDirProfileHandler::setRemoteRule() semantic: now each rule and
its sub-rules are also check for relevant conversion settings. Because
there is no session where these settings could be stored, some code had
to be copied from TSyncSession::checkRemoteSpecifics(). For the sake of
completeness, all settings which do not apply are still in the patch
as lines that were commented out.
Because setRemoteRule() is not used outside of these two script macros
(clarified in a comment), this change has no negative effect on
syncing.
|
|
pointer
It is a bit confusing that TMimeDirProfileHandler::getOptionsFromDatastore()
depends on fRelatedDatastoreP to get a session pointer when it also
could use its own getSession(). The reason is that when running as part
of a script, fRelatedDatastoreP is NULL and thus copying the wrong
options for the remote peer from the session is avoided.
This patch clarifies that and replaces the many calls to
fRelatedDatastoreP->getSession() with a local variable. It also checks
that the result really isn't NULL.
|
|
TBinfileImplClient::SelectProfile() checked whether a server URL was
configured. Such a URL is not needed in some cases, for example when
using a pre-configured transport, as in SyncEvolution's local sync
(Synthesis client <-> Synthesis server communicating via pipes).
Setting a dummy URL would have worked around the issue, but is not
the right solution. Removed the check instead.
|
