Home > Event, FOSS, ownCloud > A week for csync

A week for csync

On Friday I arrived back from Berlin where I had the pleasure to work with my great colleague Danimo and our friends from Woboq, Markus and Olivier, in the Woboq Headquarter in Berlin Kreuzberg for a week.

We thought that it might be fun to work together on csync, our sync engine under the hood of the ownCloud client. There were some issues that should be fixed and on the way we cleaned and improved quite some code in csync.

Here are some things we worked on:

  • We added a function that lets the program that uses the csync library pass arbitrary module parameters to the backend module. That way its more easy to steer the behaviour of the ownCloud modules from the calling app.
  • Error handling was improved, ie. if an http error happens, csync works errno based error reporting. We added custom errnos because not all error cases with http can be mapped to system errnos.
  • Formerly the csync ownCloud module was spooling files through an additional temporar file on client side. That step is skipped now which results in performance improvements as well as in more clean code.
  • We were able to reduce the number of HTTP requests that go over the wire even more. For example to check if there are changes on server side, now there is exactly one http propfind required. Also if files have to be synced, we could save some HTTP requests by improving caching of some requests.
  • Andreas recently changed the logging system in csync upstream master branch. We merged that back and now do not longer need the log4c framework. One build dependency less and a nice new logging framework.
  • Other bugs were fixed, such as a potential crash if a folder as deleted during it is synced, SSL handling shortcomings, code streamlining in handling compressed data streams and more.
  • We finalized a patch that uniforms the utf8 representations of characters over all platforms. That will fix problems we saw especially with MacOS and special filenames.

Ah, yes, we also did other things, more related to the ownCloud client. Danimo managed to implement a cross platform filesystem watcher class that is able to fulfill our requirements. That obsoletes polling for changes on the local file system, one of the most popular enhancement requests.

And finally there now is a API in csync thats reports file transmission progress if a callback is installed accordingly. So the client hopefully soon will tell ya what it’s doing for you. Also appreciated I guess…

Last but not least we added code to use QtKeyChain, a cross platform password storage library that stores password encrypted. For example on Linux QtKeyChain connects to kwallet. QtKeyChain was provided by
Frank Osterfeld, thanks a lot for that contribution.

Quite some stuff for a short week, note that stuff that fills a short line in this blog can be quite nifty to investigate, implement and test. Not everything is stable, polished and properly integrated but it was a great and productive week. The next release of ownCloud Client will be a nice one.

And sice you can not always work, we had a nice dinner at a very cool italian restaurant. We met with other ownCloud employees located in Berlin, Arthur and Georg. Fun 🙂 And Berlin, yes, a great place to be, but finally I appreciated to arrive back to my snow covered home.

Many thanks to Olivier and Markus for hosting us and for the nice week.

  1. Andre
    December 9, 2012 at 13:54

    Thats great news! I’m curious about the cross-platform filesystem watcher, is there an issue / ticket or something about it?

  2. Ben
    December 9, 2012 at 14:02

    Hello! I’m very interested in the “cross platform filesystem watcher class” – because my approach on windows using the NTFS change journal doesn’t work reliably.

    Could you please post a link to that code? Thanks!

  3. December 10, 2012 at 09:29

    Thanks a lot for your nice words Klaas. Let’s do it again next year 🙂

    @Andre @Ben: https://github.com/owncloud/mirall/tree/master/src/mirall -> folderwatcher_*

  4. December 10, 2012 at 09:40

    @Ben: https://github.com/owncloud/mirall/blob/master/src/mirall/folderwatcher_win.cpp does the trick for us. Note that the code isn’t all cleaned up yet, in particular folderwatcher.cpp itself. Also, we do not care for what exactly changed, since we need to run csync over all files anywy. You can get that information from the win32 API though.

  5. December 10, 2012 at 13:25

    nice poster of Banda Bassotti 🙂

  6. Matias
    December 17, 2012 at 16:02

    You know that after this last update with utf-8 the file sharing with non-standard characters was fixed, but when I try to download more than a file, when it creates the zip file, the non-standard characters look with interrogation symbol (?), for example a file called fundaçao.doc when it creates a zip, after unzip it looks like funda???ao.doc

    There is a way to fix this ?



  1. December 9, 2012 at 13:16
  2. December 21, 2012 at 12:03
  3. December 21, 2012 at 17:12
  4. December 22, 2012 at 00:10

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: