Splitting the repos...

[Allan Clark]

In support of Patrick's comments, FB gets around the cost of rebuilding
lots for a small change by CI and automation so that one of the stressors
that James mentions ("easier to only rebuild packages that are actually
changed") is eliminated. Indeed, coupling CI with automated test and
automated rollout means committing a code change and moving on, knowing
that if your change breaks something, you'll get poked.

That said, FB's lack of concern with build times may not correlate with
James' stressor.  I assume it's similar with Google.

Allan


On Thu, Apr 7, 2016 at 11:47 AM, Patrick Cloke <patrick at cloke.us> wrote:

> Gary,
>
> First off I'll say that splitting this out would like making building
> libpurple for Instantbird easier, so that'd be nice...
>
> It might make development harder, however. This is from my experiences
> developing on Instantbird which has components across 3 repositories:
> mozilla-central (UI core, networking, security, etc. from Firefox),
> comm-central (Instantbird and Thunderbird code), purplexpcom (libpurple and
> XPCOM interfaces to libpurple):
>
>    - It is significantly more difficult to get new people set-up if they
>    need to pull multiple repositories and can be more confusing to explain
>    things.
>    - Mass changes over multiple repositories can become harder (e.g.
>    renaming things, changing the build system, etc.).
>    - Building can become more frustrating: you need to ensure you have
>    corresponding revisions of both repositories that build together.
>    - People might update a single repository and forget to update others
>    (e.g. a Finch developer might make changes to Finch and corresponding
>    libpurple changes; but not update Pidgin for those same changes).
>
> I've also tried to run separate repositories in work environments in
> various environments (usually based on Git)...I've found they always fail
> miserably. (Keeping versions in sync without real rules and interfaces is a
> huge pain.) Things like git submodules/Mercurial sub-repos make this a
> little easier by pointing to a known "working revision", but then make it
> difficult to work on the bleeding edge (tip/master) in these systems.
>
> Some of the big companies, e.g. Facebook, use a single repository:
>
> "Our code base has grown organically and its internal dependencies are
> very complex. We could have spent a lot of time making it more modular in a
> way that would be friendly to a source control tool, but there are a number
> of benefits to using a single repository. Even at our current scale, we
> often make large changes throughout our code base, and having a single
> repository is useful for continuous modernization. Splitting it up would
> make large, atomic refactorings more difficult. On top of that, the idea
> that the scaling constraints of our source control system should dictate
> our code structure just doesn't sit well with us." -- Durham Goode,
> Siddharth P Agarwal
> https://code.facebook.com/posts/218678814984400/scaling-mercurial-at-facebook/
>
> I think that Google also does this (
> http://programmers.stackexchange.com/questions/41435/what-is-googles-repository-like
> ).
>
> Overall I'd say: think very carefully about why you're doing this and if
> it will really help anything. If you plan to do this, invest in some
> tooling around it (e.g. make it easy to update to compatible versions of
> all repositories without thinking much).
>
> --Patrick
> On 4/7/16 10:19 AM, Gary Kramlich wrote:
>
> This has been brought up in <devel at conference.pidgin.im>
> devel at conference.pidgin.im recently and I'd like to actually discuss it
> here where everyone can see it.  I've been waiting to bring it up until I
> had more of the CI stuff setup, but now is as good of a time as any.
>
> As you may or may not know, I strongly support splitting the repo into
> libpurple, pidgin, libgnt, and finch repos.  I would also like to split out
> some of our less commonly used protocol plugins (namely meanwhile and
> zephyr) to separate repos to get the dependency and build complexity down.
>
> For most developers, this means you'll have to build an libpurple and or
> libgnt separately.  However, once all of the CI stuff is setup, if you just
> want to work on pidgin or finch you shouldn't have to compile libpurple and
> libgnt separately anymore since you can pull them out of the CI artifacts.
>
> At any rate, discuss! :)
>
> Thanks,
>
> --
> Gary Kramlich <grim at reaperworld.com>
>
>
> _______________________________________________
> Devel mailing listDevel at pidgin.imhttps://pidgin.im/cgi-bin/mailman/listinfo/devel
>
>
>
> _______________________________________________
> Devel mailing list
> Devel at pidgin.im
> https://pidgin.im/cgi-bin/mailman/listinfo/devel
>



-- 
allanc at chickenandporn.com  "金鱼" http://linkedin.com/in/goldfish
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pidgin.im/pipermail/devel/attachments/20160407/9b5d0495/attachment.html>