Home > FOSS, ownCloud > What’s on in Nightly?

What’s on in Nightly?

We ownCloud Client devs were quite busy doing bug investigations and two releases the last couple of weeks. But that does not mean that nothing went into feature branches. It did, and now we can start to think of a release 1.3.0 with a couple of new features for the client.

Here is a small overview of what went into ocsync master (actually the dav branch) and into mirall master.

Note that this is alpha state software, but it’s already available for testing from our nightly builds at the ownCloud nightly repository.

If you enable the nightly package repository on your system, please be sure to use the machine for testing purposes only. Windows and Mac nightly builds can be found here to download.

csync: Big File Chunking

Handling big files over HTTP is difficult, if files grew bigger than a gigabyte, we faced multiple problems. To deal with that, we thought of splitting up big files and transmit the parts to let the server finally assemble the file again once all parts are there.

We wrote a little library to implement that, called httpbf. It is now integrated in the current development branch. You will see files big files split up into parts for transmission to the server. With that there should not be a filesize limit to work with any more, except the limits we have from the operating system etc.

Up- and Download Resume

With big file chunking, we also introduce up- and download resume. If a file transmission was interrupted and is started over again later, only the missing parts are going to be pushed again. However, we’re not yet transmitting only changed chunks for existing files yet, just to answer that question here before it comes up. That will be subject of more enhancements.

Source File Verification

Another problem with big files is that if one gets copied into the synced dir on the local machine, the copy is not finished before the sync acutally starts. That can result in useless transmissions because inconsistent data is synced.

We added code to the httpbf library that checks for the integrity of the source file. When the first chunk is transmitted, it is checked if the file has still the size and mod time from the beginning. If not, the library waits for two seconds and starts over with a check again. That way, it waits until the source file has stabilized. A final check if the source file hasn’t changed during upload is done for every file when the upload is finished.

CSync Context Reuse

There is the very unpleasant problem on some distros (namely Ubuntu and Fedora) that the ownCloud Client wastes file descriptors. We figured that this is because the system libraries on that platforms fail to unload the csync ownCloud module properly.

Version 1.2.5 contains a quick fix for that, but the real fix is to reuse the csync context within mirall and not build it up and down for every sync run as we did until now. So we developed the function csync_commit which commits the results of the sync run to the database and leaves the csync context intact for the next run. That way we just load and unload the ownCloud module on time and not every 30 seconds. The csync patch already went upstream.

Mirall: New Setup Dialog

newsetup_sThere is also the new setup dialog in the nightly builds: It is much more simplified compared to the old one, user only needs to add the url and user credentials and thats it. Also, it syncs the whole ownCloud to a local folder, just like other boxes do. That is one of the most often heard enhancement requests, and here we go.

That all I think would qualify for a good 1.3.0 release, but we need more time to fine tune,polish, test and bugfix. But we would appreciate if people could help test. Please report bugs in the client’s bugtracker.
We need to find bugs prior the release, and that happens with your help!

Have fun & Thanks a lot for testing!

About these ads
  1. eric
    April 29, 2013 at 16:00

    i’m really looking forward this version, having a slow internet connection and a low timeout and maxsize php configs (I cannot change them), chunk uploading and resume will surely help a lot! how big is each chunk?

    • dragotin
      April 29, 2013 at 16:07

      Currently a chunk is 10 MB but we could make that an configurable value. We should gather some experience first which default makes sense.

  2. April 29, 2013 at 20:24

    I was always interested in the development of OwnCloud, but today I got around installing it on my RasPi. Great thing, I found a lot of things I can use it for. Probably one of the most useful projects I’ve seen.

  3. Martin
    April 30, 2013 at 09:05

    I don’t know if this is how cloud storage is supposed to be used, but are the plans to have an preview of which files will be updated (and in which direction, Server->Local or Local->Server) and which files will be deleted?
    Maybe something like this: http://goo.gl/KEPdh

  4. May 1, 2013 at 18:44

    Hi Klaas. The way you handle details about upcoming releases is great. Transparent and valuable. Cheers to Danimo as well. Bandwith control would be great as well.

  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: