Re: IMPORTANT: Spark mailing lists moving to Apache by September 1st

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: IMPORTANT: Spark mailing lists moving to Apache by September 1st

andy
Administrator
Per Matei's suggestion, I've set up two nabble archive lists, one to archive the apache dev list and one to archive the apache user list.

user list archive: http://apache-spark-user-list.1001560.n3.nabble.com

Between these and whatever solution we end up with for the google group mirrors, we should have decent enough alternatives to reading via the apache list archives going forward.


On Thu, Dec 19, 2013 at 11:09 PM, Matei Zaharia <[hidden email]> wrote:
Yes, I agree that we should close down the existing Google group on Jan 1st. While it’s more convenient to use, it’s created confusion. I hope that we can get the ASF to support better search interfaces in the future too. I think we just have to drive this from within.

The Google Group should be a nice way to make the content searchable from the web. We should also see what it takes to make it mirrored on Nabble (http://www.nabble.com). I’ve found a lot of information about other projects there, and other Apache projects do use it.

Matei

On Dec 19, 2013, at 10:49 PM, Andy Konwinski <[hidden email]> wrote:

I've set up two new unofficial google groups to mirror the Apache Spark user and dev lists:


Basically these lists each subscribe to the corresponding Apache list.

They do not allow folks to subscribe directly to them. Getting emails from the Google Group would offer no advantages that I can think of and we really want to encourage folks to sign up for the official mailing list instead.

The lists do allow the public to send email to them, which I think might be necessary since the "from:" field for all emails that get distributed via the Apache mailing list is set to the author of the email.

I think this might be a great compromise. At least we can try this out and see how it goes.

Matei, can you confirm that Jan 1 is the date we want to turn off the existing spark-users google group?

We could consider using the existing spark-developers and spark-users google groups instead of the two new ones I just created but I think that it is much more obvious to have the lists include the word mirror in their names.

The dev list mirror seems to be working, because I see the last couple emails from this thread in it already. I'll confirm and ensure that the user list mirror is working too.

Thoughts?

Andy

P.S. Thanks to Patrick for suggesting this to me originally.

On Thu, Dec 19, 2013 at 8:46 PM, Aaron Davidson <[hidden email]> wrote:
I'd be fine with one-way mirrors here (Apache threads being reflected in Google groups) -- I have no idea how one is supposed to navigate the Apache list to look for historic threads.


On Thu, Dec 19, 2013 at 7:58 PM, Mike Potts <[hidden email]> wrote:
Thanks very much for the prompt and comprehensive reply!  I appreciate the overarching desire to integrate with apache: I'm very happy to hear that there's a move to use the existing groups as mirrors: that will overcome all of my objections: particularly if it's bidirectional! :)


On Thursday, December 19, 2013 7:19:06 PM UTC-8, Andy Konwinski wrote:
Hey Mike,

As you probably noticed when you CC'd spark-de...@googlegroups.com, that list has already be reconfigured so that it no longer allows posting (and bounces emails sent to it).

We will be doing the same thing to the [hidden email] list too (we'll announce a date for that soon).

That may sound very frustrating, and you are *not* alone feeling that way. We've had a long conversation with our mentors about this, and I've felt very similar to you, so I'd like to give you background.

As I'm coming to see it, part of becoming an Apache project is moving the community *fully* over to Apache infrastructure, and more generally the Apache way of organizing the community.

This applies in both the nuts-and-bolts sense of being on apache infra, but possibly more importantly, it is also a guiding principle and way of thinking.

In various ways, moving to apache Infra can be a painful process, and IMO the loss of all the great mailing list functionality that comes with using Google Groups is perhaps the most painful step. But basically, the de facto mailing lists need to be the Apache ones, and not Google Groups. The underlying reason is that Apache needs to take full accountability for recording and publishing the mailing lists, it has to be able to institutionally guarantee this. This is because discussion on mailing lists is one of the core things that defines an Apache community. So at a minimum this means Apache owning the master copy of the bits.

All that said, we are discussing the possibility of having a google group that subscribes to each list that would provide an easier to use and prettier archive for each list (so far we haven't gotten that to work).

I hope this was helpful. It has taken me a few years now, and a lot of conversations with experienced (and patient!) Apache mentors, to internalize some of the nuance about "the Apache way". That's why I wanted to share.

Andy

On Thu, Dec 19, 2013 at 6:28 PM, Mike Potts <[hidden email]> wrote:
I notice that there are still a lot of active topics in this group: and also activity on the apache mailing list (which is a really horrible experience!).  Is it a firm policy on apache's front to disallow external groups?  I'm going to be ramping up on spark, and I really hate the idea of having to rely on the apache archives and my mail client.  Also: having to search for topics/keywords both in old threads (here) as well as new threads in apache's (clunky) archive, is going to be a pain!  I almost feel like I must be missing something because the current solution seems unfeasibly awkward!


--
You received this message because you are subscribed to the Google Groups "Spark Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to spark-users...@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.




--
You received this message because you are subscribed to the Google Groups "Spark Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "Spark Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].

For more options, visit https://groups.google.com/groups/opt_out.

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: IMPORTANT: Spark mailing lists moving to Apache by September 1st

andy
Administrator
Unfortunately, I don't see that as an option for either these nabble lists or the google groups.

For the google groups, we we could subscribe the old spark google groups to the apache lists (instead of using the new mirror groups I created).

However the tradeoff is that as far as I can tell, we wouldn't be able to have posting disabled or even subscriber only for those groups (since the emails from the apache lists would then get bounced). In this case, I'm concerned that if it remains possible to post to the google groups, then people will continue posting to them, which would be confusing to everybody.

We could rename the old groups to make it clear they are the mirrors, however then the people who are subscribed to them and have rules setup for dealing with them would have to change their rules.

Finally, we could unsubscribe everybody from the old google groups and then rename them to have the word mirror in them which would preserve history and be no worse than if they just became silent and activity moved to the new mirror groups i created (which as I mentioned, must allow posting by anybody). So I guess this last option is what I propose we do.

Andy


On Sun, Dec 22, 2013 at 3:51 PM, Mike Potts <[hidden email]> wrote:
Thanks!!  Is is straightforward to pre-populate it with all the existing threads from apache?

--
You received this message because you are subscribed to the Google Groups "Spark Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: IMPORTANT: Spark mailing lists moving to Apache by September 1st

andy
Administrator

It would be a good thing, I'm just not sure how to achieve it, or if it's possible. AFAIK, you cannot simply subscribe the lists to each other. Have you heard of a setup like this being used before?

On Dec 22, 2013 7:12 PM, "Mike Potts" <[hidden email]> wrote:
Just a thought: if the mirroring was bidirectional (between apache lists and google groups) -- meaning that you could read/write via either channel and everything would be mirrored in both groups -- would that be a bad thing?  The apache lists would be comprehensive and available as a first-class communications channel; and so would the google groups: I would have thought the benefits of maintaining both channels (fully mirrored) would outweigh any drawbacks... but again I'm probably missing something :)

--
You received this message because you are subscribed to the Google Groups "Spark Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: IMPORTANT: Spark mailing lists moving to Apache by September 1st

Patrick Wendell
In reply to this post by andy
Hey Andy - these Nabble groups look great! Thanks for setting them up.


On Tue, Dec 24, 2013 at 10:49 AM, Evan Chan <[hidden email]> wrote:
Thanks Andy, at first glance nabble seems great, it allows search plus posting new topics, so it appears to be bidirectional.    Now just have to register an account on there.


On Sun, Dec 22, 2013 at 2:47 PM, Andy Konwinski <[hidden email]> wrote:
Per Matei's suggestion, I've set up two nabble archive lists, one to
archive the apache dev list and one to archive the apache user list.

user list archive: http://apache-spark-user-list.1001560.n3.nabble.com
dev list archive: http://apache-spark-developers-list.1001551.n3.nabble.com

Between these and whatever solution we end up with for the google group
mirrors, we should have decent enough alternatives to reading via the
apache list archives going forward.


On Thu, Dec 19, 2013 at 11:09 PM, Matei Zaharia <[hidden email]>wrote:

> Yes, I agree that we should close down the existing Google group on Jan
> 1st. While it’s more convenient to use, it’s created confusion. I hope that
> we can get the ASF to support better search interfaces in the future too. I
> think we just have to drive this from within.
>
> The Google Group should be a nice way to make the content searchable from
> the web. We should also see what it takes to make it mirrored on Nabble (
> http://www.nabble.com). I’ve found a lot of information about other
> projects there, and other Apache projects do use it.
>
> Matei
>
> On Dec 19, 2013, at 10:49 PM, Andy Konwinski <[hidden email]>
> wrote:
>
> I've set up two new unofficial google groups to mirror the Apache Spark
> user and dev lists:
>
> https://groups.google.com/forum/#!forum/apache-spark-dev-mirror
> https://groups.google.com/forum/#!forum/apache-spark-user-mirror
>
> Basically these lists each subscribe to the corresponding Apache list.
>
> They do not allow folks to subscribe directly to them. Getting emails from
> the Google Group would offer no advantages that I can think of and we
> really want to encourage folks to sign up for the official mailing list
> instead.
>
> The lists do allow the public to send email to them, which I think might
> be necessary since the "from:" field for all emails that get distributed
> via the Apache mailing list is set to the author of the email.
>
> I think this might be a great compromise. At least we can try this out and
> see how it goes.
>
> Matei, can you confirm that Jan 1 is the date we want to turn off the
> existing spark-users google group?
>
> We could consider using the existing spark-developers and spark-users
> google groups instead of the two new ones I just created but I think that
> it is much more obvious to have the lists include the word mirror in their
> names.
>
> The dev list mirror seems to be working, because I see the last couple
> emails from this thread in it already. I'll confirm and ensure that the
> user list mirror is working too.
>
> Thoughts?
>
> Andy
>
> P.S. Thanks to Patrick for suggesting this to me originally.
>
> On Thu, Dec 19, 2013 at 8:46 PM, Aaron Davidson <[hidden email]>wrote:
>
>> I'd be fine with one-way mirrors here (Apache threads being reflected in
>> Google groups) -- I have no idea how one is supposed to navigate the Apache
>> list to look for historic threads.
>>
>>
>> On Thu, Dec 19, 2013 at 7:58 PM, Mike Potts <[hidden email]> wrote:
>>
>>> Thanks very much for the prompt and comprehensive reply!  I appreciate
>>> the overarching desire to integrate with apache: I'm very happy to hear
>>> that there's a move to use the existing groups as mirrors: that will
>>> overcome all of my objections: particularly if it's bidirectional! :)
>>>
>>>
>>> On Thursday, December 19, 2013 7:19:06 PM UTC-8, Andy Konwinski wrote:
>>>
>>>> Hey Mike,
>>>>
>>>> As you probably noticed when you CC'd [hidden email],
>>>> that list has already be reconfigured so that it no longer allows posting
>>>> (and bounces emails sent to it).
>>>>
>>>> We will be doing the same thing to the [hidden email] list
>>>> too (we'll announce a date for that soon).
>>>>
>>>> That may sound very frustrating, and you are *not* alone feeling that
>>>> way. We've had a long conversation with our mentors about this, and I've
>>>> felt very similar to you, so I'd like to give you background.
>>>>
>>>> As I'm coming to see it, part of becoming an Apache project is moving
>>>> the community *fully* over to Apache infrastructure, and more generally the
>>>> Apache way of organizing the community.
>>>>
>>>> This applies in both the nuts-and-bolts sense of being on apache infra,
>>>> but possibly more importantly, it is also a guiding principle and way of
>>>> thinking.
>>>>
>>>> In various ways, moving to apache Infra can be a painful process, and
>>>> IMO the loss of all the great mailing list functionality that comes with
>>>> using Google Groups is perhaps the most painful step. But basically, the de
>>>> facto mailing lists need to be the Apache ones, and not Google Groups. The
>>>> underlying reason is that Apache needs to take full accountability for
>>>> recording and publishing the mailing lists, it has to be able to
>>>> institutionally guarantee this. This is because discussion on mailing lists
>>>> is one of the core things that defines an Apache community. So at a minimum
>>>> this means Apache owning the master copy of the bits.
>>>>
>>>> All that said, we are discussing the possibility of having a google
>>>> group that subscribes to each list that would provide an easier to use and
>>>> prettier archive for each list (so far we haven't gotten that to work).
>>>>
>>>> I hope this was helpful. It has taken me a few years now, and a lot of
>>>> conversations with experienced (and patient!) Apache mentors, to
>>>> internalize some of the nuance about "the Apache way". That's why I wanted
>>>> to share.
>>>>
>>>> Andy
>>>>
>>>> On Thu, Dec 19, 2013 at 6:28 PM, Mike Potts <[hidden email]> wrote:
>>>>
>>>>> I notice that there are still a lot of active topics in this group:
>>>>> and also activity on the apache mailing list (which is a really horrible
>>>>> experience!).  Is it a firm policy on apache's front to disallow external
>>>>> groups?  I'm going to be ramping up on spark, and I really hate the idea of
>>>>> having to rely on the apache archives and my mail client.  Also: having to
>>>>> search for topics/keywords both in old threads (here) as well as new
>>>>> threads in apache's (clunky) archive, is going to be a pain!  I almost feel
>>>>> like I must be missing something because the current solution seems
>>>>> unfeasibly awkward!
>>>>>
>>>>>
>>>>> --
>>>>> You received this message because you are subscribed to the Google
>>>>> Groups "Spark Users" group.
>>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>>> an email to [hidden email].
>>>>>
>>>>> For more options, visit https://groups.google.com/groups/opt_out.
>>>>>
>>>>
>>>>
>>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Spark Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [hidden email].
>
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>  --
> You received this message because you are subscribed to the Google Groups
> "Spark Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [hidden email].
>
> For more options, visit https://groups.google.com/groups/opt_out.
>



--
--
Evan Chan
Staff Engineer
[hidden email]  | 


--
You received this message because you are subscribed to the Google Groups "Spark Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.

Loading...