Update (2018-07-31): Libravatar is not going away

Here is a place where people who are interested in taking over Libravatar can coordinate.

Your contribution is welcome!

A group of people is working on continuing the libravatar development (code and service).

Please join the communication channels:

IRC meetings

2018-10-28 (Upcoming)

Everyone interested in supporting the libravatar project (code or service) is welcome!

2018-10-14

  • participants: clime, fmarier, sumpfralle, tleguern

Review of the last weeks

  • clime: new implementation (for testing) is available on fedora infrastructure:
    • http://libravatar.fedorainfracloud.org/
    • http://libravatar-stg.fedorainfracloud.org/
  • fmarier & sumpfralle: mail and backup was moved

Data migration

  • before we can switch to the new implementation, we will need to migrate the existing data
  • this may partly be done via Django migrations (with some manual fiddling)
  • the current filesystem based storage (images?) need to be moved to the database
  • maybe ofalk or clime could do this, but volunteers are certainly welcome!
  • sumpfralle will send a request for activity / volunteers to the mailing list

Mail handling

  • the mail domain is now handled https://systemausfall.org/ (via sumpfralle)
  • existing forwards are put in place
  • we had a longer discussion about whether we want to allow private email addresses (john.doe@libravatar.org) fo contributors
  • there was a slight tendency towards yes (with some reluctance expressed by others)
  • sumpfralle will send a summary of the state and how to use it to the mailing list

Project name

  • we continued a bit on the discussion, whether a name change would be necessary/helpful for some reason
  • clime will take a look at the details and send a follow-up to the mailing list

2018-09-30

  • participants: clime, ij, ofalk, sumpfralle, tleguern

Review of the last weeks

  • ofalk: some more progress with the code
  • clime: no significant progress with the deployment - this will change in the next week

Project name

  • ofalk raised the issue, that the fedora people (to be responsible for hosting the service) raised concerns regarding the licensing / trademark of the name "libravatar" and its logo
    • details: https://pagure.io/design/issue/613
  • ofalk will open this discussion on the mailing list

Regular IRC meetings

  • we discussed, whether we want to continue having periodic IRC meetings
  • most people expressed their preference for keeping the biweekly IRC meetings
    • no objections were raised
  • so we will continue to meet every two weeks on Sunday evening at 19 UTC

Who is "we"?

  • we discussed, whether a legal entity would be required for the project
    • all participants preferred to avoid this
    • ofalk would be willing to register the domain (and thus being the only publicly identifiable person responsible for the service)
      • people expressed their willingness to share the related costs
  • proposed self description of the group: We are the group of people involved with developing the software and maintaining the service currently known as "libravatar". This entity does not have a legal status. We are just a group of people. We use a mailinglist and an IRC channel for discussions and decisions.

2018-09-02

  • participants: falko, fmarrier, nipos, sumpfralle, tleguern

Infrastructure

  • we will use the OpenStack infrastructure from Fedora (instead of OpenShift)

Current state of development =

  • Design:
    • Two links in the header nav link to nowhere
  • image size issue (size=0 and size=) is fixed
  • structure is similar to the previous code
  • original pictures are stored in the database
  • resized variations are rendered on demand
    • caching/something may follow (at the appropriate place)
  • the lack of a filesystem-based storage will be challenging for potential mirrors
    • we will discuss this (and potential changes), after the new code is deployed

Data migration

As soon as we think that the new implementation is ready, we can migrate the data.

  • prepare export and import scripts
  • migrate everything, except for the passwords
    • remind people of the existence of libravatar :)
    • improve password hash quality
    • allow cleanup stale/unused accounts after a while

Migration process

  • send a mail in advance: inform about the change to a new platform and the new group of "us"
  • migrate the data; shutdown the original host (thanks, ij!)
  • send a mail requesting users to set a new password

Mail-Domain

  • sumpfralle offered to forward mails for the few @libravatar.org accounts to the respective people
    • dev, tls, support, postmaster, ...
  • outgoing mail is handled by the server itself

Backup

  • currently fmarrier provides the backup space (< 10GB)
  • sumpfralle offered space with ssh access

2018-08-19

  • Participants: clime, nipos, sumpfralle

Review

What happened during the last two weeks?

  • fmarier moved the libravatar service the new VM provided by ij (2018-08-13)
  • nipos got the new libravatar code running (preparing his design work)

Hosting the service on Fedora infrastructure

  • clime communicated with Fedora people
  • we will be able to host the libravatar on their infrastructure
  • the host will be managed by the Fedora infrastructure team
  • probably clime and oliver will have administrative access and will care for the hosting details of libravatar for the forseeable future
    • administrative access by non-Fedora people is probably not trivial/possible (it is unclear, if there is demand for that)
  • The steps for the migration of the current hosting (with the old implementation) to Fedora infrastructure are the following:
    • prepare an ansible playbook for setting up the system for the service
    • run a one-time data migration from ij's VM
    • do a bit of tests with the new deployment
    • trigger a final data migration from ij's VM and switch the DNS entries to the new IP of the Fedora host
  • clime will work on this
  • Please note: we were only three people discussing this, thus the plan currently does not rest on broad explicit acceptance. Please raise your voice, if you see problems with this strategy.

Hosting of git repositories

We will probably use git repositories for hosting the code, the ansible playbook and maybe more.

  • clime suggested pagure.io
  • if we use the Fedora hosting, the ansible playbook would probably be part of https://infrastructure.fedoraproject.org/cgit/ansible.git
    • sync with pagure.io should be no problem (for issue reporting and so on)
  • other hosting services (running on free software) would be acceptable, too

2018-08-05

  • Participants: fmarier, ij, kumy, nipos, opal, sumpfralle, tklk, tleguern

Review

What happened during the last week?

  • new mirror set up by Asako
  • new VM is up with double RAM (compared to the current one)
    • fmarier has ssh access to the vm (root) - he will install the service and migrate data in the next days

GDPR

What do we need to do with regards to handling of personal data. Can we migrate the old data?

Which data is currently stored?

  • email adresses: for password resets - although emails can be hashed and still be used for this purpose (even though emails can be removed, this might not be great UX as users don't know which accounts they are assigning an avatar to)
  • username: for logging in
  • uploaded photos: obvious primary use of the service
  • linked OpenIDs
  • login cookie
  • we used to store ip addresses but it's all gone
  • IP adresses in server logs
  • On mirrors, mod_removeip is enabled -> no IPs are logged
  • On master access logs are stored for 10 days (currently with ips)
  • We do not store any IP address starting at 05/08/18 19:26 UTC - in 10 days, no IP remains in the logs

Should / could we do something at the moment?

  • suggestion of mass email (no consent given by users to have mass emails, so I'm against this) to me this seems like a special case and falls under the category of "important email" in order to follow legislation
  • add checkbox for new accounts that they've accepted TOS/Privacy policy
  • add terms of use/privacy policy
  • We have support for data deletion and export since first release (as wanted by the GDPR).
  • We miss a "privacy policy" text.
    • opal offered to write a text for libravatar.

Summary: we think, we are currently complying with law requirements. We will take a closer look at the new implementation with regards to opt-in and privacy policy.

Evaluate the state of the new implementation

Neither Oliver nor clime present, thus we skipped this topic.

Discuss the hosting options for the new implementation

  • Where does code hosting go? does it stay with launchpad (peferably pagure to move everything to fedora infra)
  • Domain transfer- where does it go? (preferably fedora, instead of individual to reduce bus factor)
  • DigitalOcean would probably sponsor something (for 1 year only?)
  • smurf offered hosting too
  • No final decisions were made. Maybe collect options and pros/cons in the wiki and via the mailing list?

Discuss about new design concept

Nipos proposed a design concept: https://codepen.io/niklasposlovski/full/RByopx/

Various details

  • "Mirrors may not require SNI": https://wiki.libravatar.org/how_to_add_a_mirror_slave_to_the_mirror_master/
    • probably we should lift this requirement? (only elinks seems to lack support for SNI)
  • Mirrors running in a Docker container for easier setup?
  • Mirrors are tied to the Debian package implementation (crons), what about other distros? (Alpine, Arch)
  • Drop HTTP support to HTTPS only - HSTS?

2018-07-29

  • Participants (IRC nicknames): clime, ij, fmarier, kumy, nipos, ofalk, sumpfralle, tleguern
  • Challenges of our endeavour:
    • organize a new development group / process
    • organize a new hosting setup / group
  • Software implementation options:
    • A) improve the current implementation
    • B) Oliver's new Django-based implementation
    • C) Matti's implementation in golang
    • D) tleguern's C-based implementation (targetted at self-hosting people)
    • Conclusion: we decided for B
  • Discussion topics:
    • if/how to keep the data of existing users
    • handling of GDPR issues when transferring the user's data to a new operator team
    • different hosting options (long-term)
    • future communication channels
  • Goals:
    • Goal #1: Do we want to continue the service longer than September this year?
      • everyone agrees
    • Goal #2: Do we want to continue offering the service past September with the current dataset of users? (GDPR issues will have to be solved for that)
      • majority agrees
        • clime is concerned about the service downtime (avatars being available for all users at all times) during data migration
  • Migration plan: fmarier will help migrate the current setup (code + data) to a new VM and admits to operate that for up to two months longer (end of November). Meanwhile we are rushing to resolve GDRP issues and migrate the data to the new implementation and get that live as soon as possible. This will relieve fmarier from the service.
  • Actions:
    • fmarier and ij will migrate the service as-is to a new VM
    • ofalk will review potentially missing features of his implementation
    • nipos will try to prepare something for the new web interface
  • Communication & Meetings:
    • everyone interested should subscribe to the mailing list: https://launchpad.net/~libravatar-fans
    • next IRC meeting: 2018-08-05, 19:00 UTC, #libravatar at freenode
      • afterwards: biweekly meetings
  • Next issues to be discussed:
    • GDPR: can we migrate the old data?
    • Evaluate the state of the new implementation.
    • Discuss the hosting options for the new implementation.

Contacts / Offers of Help

Please leave some contact details so that others can reach out to you.


I can offer hosting, domain, CDN for free. showfom@gmail.com

Can we get some statistics about the service now? Both in terms of the infrastructure (assuming infra support won't continue) and in terms of what is required to keep it going? Do we know how many avatars are served? stored? etc? (bex@pobox.com)

I second this. I would like to see this service keep going if possible. How many servers are currently in production and what is the monthly cost to host them? I have over 15 years experience as a server admin including time in the hosting industry and would love to contribute. Email is wattersm@watters.ws

I also would like to keep the service alive. I can offer free hosting on dedicated ipv4 and ipv6. I'm currently trying to deploy a private instance of the project using docker. libravatar@kumy.net

I can offer free hosting for the service. smurf@debian.org


Stats

As of 2018-04-03, the storage requirements are:

  • Application server:

    • 1 GB General Purpose v1 Rackspace cloud server
    • Number of user accounts: 6655
    • Number of unique avatars: 6919 (355 MB)
  • Static image server:

For the month of March 2018, the traffic looked like this:

  • Application server:

    • Bandwidth: 328.31 MB in awstats (45 GB in Rackspace billing)
    • Hits: 43,823
  • Static image server:

    • Bandwidth: 18.99 GB in awstats (68 GB in Rackspace billing)
    • Total hits: 11,191,373
    • Redirects (to Gravatar): 9,605,127

Current stack

  • OS: Debian 8.11 (jessie)
  • Framework: Django 1.7.11
  • Database: PostgreSQL 9.4
  • Webserver: Apache 2.4.10
  • Queue: Gearman 1.0.6

Cost

The cost (currently covered by Rackspace through their OSS program) for the month of March 2018 was $70 USD:

  • Cloud Bandwidth: $13.50
  • Cloud Servers: $56.84

External accounts

In addition to the domain names (libravatar.org and libravatar.com), the following accounts could be transferred to the new team:


Basically I'm willing to help to keep this service up & running and would like to contribute a VM or some disk space and bandwidth by running a mirror. No Django experience though, but slightly able to read Python, but no programmer at all. Contact me at ij@2017.bluespice.org or @ingoj on Twitter.


Digital Ocean

DigitalOcean would be a good place to move this service. A couple of $10 droplets could easily handle the load. The question is who would you want to own the servers?

The email to reach out for potential DO open source sponsorship is opensource@digitalocean.com

They offered $200 to host two droplets for one year.


DO also ( I believe ) will contribute resources to open source projects that request it.

Has anyone formulated a plan at all to update Libravatar to Python 3 and migrate away from unsupported libraries? zach@mailcan.com


I have setup a new repo here: https://pagure.io/libravatar2 to modernize/rewrite libravatar to use the latest available tooling. Please concact me if you would like to help.

clime@redhat.com


I am here to help anyway I can, refactoring code, hosting. Please reach out sadin@fedoraproject.org


I published today a server-side implementation in C I started three years ago, available here: https://github.com/Aversiste/libravatar.cgi. It is a CGI program currently implementing the size, default and forcedefault flags with allowed default values of '404', 'mm' and 'blank'.

This is not finished and not a suitable replacement for the current stack as it uses the file system directly but it is good enough for self-hosting. I use it at https://avatars.bouledef.eu/index

I am available for any discussions related to the shutdown and the migration though - tleguern@bouledef.eu


I want to keep this great project alive.I'm the developer of the Mastodon client Halcyon which I took over when the original developer quit his work and deleted his repository.I successfully made it popular again and added many new functions so far.I hope that I can archieve this goal a second time with Libravatar.I don't know the server side languages used here (I'm an PHP developer) but I can modernize the client side and with the support of a great community who will hopefully pay for the servers I can also manage the hosting.I want to keep things open and I think it's bad when everyone has to change to the closed source Gravatar.Instead I want to keep an open source avatar hoster which hopefully will some time in the future federated.I'm always here to help - Email: ni.pos@yandex.com Jabber: bw5rws@jabber.lima-city.de Matrix: @nipos:avareborn.de And I'm also in the #libravatar IRC Channel which could be helpful for coordinating the future of this project.


@all, but especially to @clime@redhat.com. I've created a local repo on my Gitlab for the moment, rewrote most of the code for Django 2 and deployed a first more or less working copy to OpenShift. Questions, please don't hesitate to contact me at: oliver@linux-kernel.at Regarding the question about libravatar and Python 3. I didn't touch the lib yet, but that will also happen, yes. I think the first, important move, was to upgrade the backend to Django 2. Next is to modernize the frontend (I'm thinking about Bootstrap... And no, no Angular, React, Vue, ...). I really hope that nobody else is working on this and is as far as I am already. :-) Source code can be found here.


Catalyst Cloud would be happy to host a mirror - Andrew Ruthven, puck@catalystcloud.nz