Splitting the repos...

[Gary Kramlich]

On Thu, Apr 7, 2016 at 1:47 PM, Patrick Cloke <patrick at cloke.us> wrote:

> Gary,
>
> First off I'll say that splitting this out would like making building
> libpurple for Instantbird easier, so that'd be nice...
>
> It might make development harder, however. This is from my experiences
> developing on Instantbird which has components across 3 repositories:
> mozilla-central (UI core, networking, security, etc. from Firefox),
> comm-central (Instantbird and Thunderbird code), purplexpcom (libpurple and
> XPCOM interfaces to libpurple):
>
 Nice I didn't realize there was a purple-xpcom :)  Anyways, how is this
different than how Pidgin has a ton of external dependencies?  I'm not
trying to justify it, just wondering what makes this different.

>
>    - It is significantly more difficult to get new people set-up if they
>    need to pull multiple repositories and can be more confusing to explain
>    things.
>
> With packages everywhere (even dev ones) (see below) that should help with
getting people setup.  It really just is another dependency.

>
>    - Mass changes over multiple repositories can become harder (e.g.
>    renaming things, changing the build system, etc.).
>
> Changing build systems is and should be a per repo/project thing.  For
example, Pidgin uses autotools and depends on GPlugin which uses CMake.

>
>    - Building can become more frustrating: you need to ensure you have
>    corresponding revisions of both repositories that build together.
>
>  Again, hoping to solve this with CI/CD.

>
>    - People might update a single repository and forget to update others
>    (e.g. a Finch developer might make changes to Finch and corresponding
>    libpurple changes; but not update Pidgin for those same changes).
>
> Not all feature will make sense in all UI's; even if they do, I don't
think one should block another from being released.

> I've also tried to run separate repositories in work environments in
> various environments (usually based on Git)...I've found they always fail
> miserably. (Keeping versions in sync without real rules and interfaces is a
> huge pain.) Things like git submodules/Mercurial sub-repos make this a
> little easier by pointing to a known "working revision", but then make it
> difficult to work on the bleeding edge (tip/master) in these systems.
>
Every time I've tried external (svn), submodules (git), sub-repos (hg),
it's turned out horribly.  I'm hoping to avoid most of this with CI/CD
which I get into more details below.

> Some of the big companies, e.g. Facebook, use a single repository:
>
> "Our code base has grown organically and its internal dependencies are
> very complex. We could have spent a lot of time making it more modular in a
> way that would be friendly to a source control tool, but there are a number
> of benefits to using a single repository. Even at our current scale, we
> often make large changes throughout our code base, and having a single
> repository is useful for continuous modernization. Splitting it up would
> make large, atomic refactorings more difficult. On top of that, the idea
> that the scaling constraints of our source control system should dictate
> our code structure just doesn't sit well with us." -- Durham Goode,
> Siddharth P Agarwal
> https://code.facebook.com/posts/218678814984400/scaling-mercurial-at-facebook/
>
> I think that Google also does this (
> http://programmers.stackexchange.com/questions/41435/what-is-googles-repository-like
> ).
>
This doesn't map 1:1 to the pidgin repo as it stands today.  We have 1
build system for 4 projects that can stand alone.

> Overall I'd say: think very carefully about why you're doing this and if
> it will really help anything. If you plan to do this, invest in some
> tooling around it (e.g. make it easy to update to compatible versions of
> all repositories without thinking much).
>
The tooling is most of the CI work that I'm doing.  I haven't spoke much
about it publically because, well right now the time is better spent making
it work :)  Anyways, my goal with CI is to actually make it CI/CD.  I still
have a ton of stuff to do on the Pidgin side, but the GPlugin stuff all
builds packages per build job.  In other words, who needs nighlighties :)
 That said, nightlies and releases would already be packaged for everyone.
I'm looking at some cloud based repository providers (bintray.com and
packagecloud.io) for making these packages available to everyone once their
built.

This should actually make it easier for Instabird as well, since you now
longer need to compile libpurple, you can just say you need this version
from this repo.  That said, once the pidgin stuff is setup in CI/CD I'd be
willing to pull in purple-xpcom and whatever else we can help you
build/test.

For the record, we're using Bamboo On Demand with a Open Source (gratis)
license from Atlassian.  It can be found at https://pidgin.atlassian.net.
The GPlugin builds are messed because I'm trying to get more agents running
on my machine by running them in Docker containers.  The entire CI/CD setup
is a bit confusing and I will be documenting it in the near future.

--Patrick
>

Thanks,

--
Gary Kramlich <grim at reaperworld.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pidgin.im/pipermail/devel/attachments/20160407/163930b2/attachment-0001.html>