Conversation with #fedora-workstation at Tue 24 Mar 2015 08:01:15 AM EDT on sgallagh/FreeNode@fedorashell.rdu.redhat.com (irc)
(08:01:15 AM) Topic for #fedora-workstation set by stickster at 02:57:12 PM on 03/16/2015
(08:01:51 AM) You are now known as sgallagh
(09:26:16 AM) stickster: http://blog.linuxgrrl.com/2015/03/24/enabling-new-contributors-brainstorm-session/ is pretty short notice, but I can represent the WG there
(09:52:26 AM) sgallagh: stickster: Have you or anyone else from the Workstation WG opened the discussion with the Env/Stacks WG about what repos should end up in the Playground?
(09:52:38 AM) sgallagh: It seems to me that we probably want to get those sorted before Beta Freeze.
(09:53:54 AM) stickster: sgallagh: I have not, is there a page for current candidates? Or are we basically picking from the entire universe of COPRs to e.g. take advantage of the new disabled repo feature?
(09:54:40 AM) sgallagh: stickster: Last I checked, the list was still empty.
(09:54:56 AM) sgallagh: You are NOT allowing every repo in COPR to be searched, though.
(09:55:21 AM) sgallagh: Playground is exactly that; it's going to be a Fedora package that can be installed by default that will have enable_metadata repos in it
(09:55:28 AM) stickster: Right, this would be selecting *a* COPR (or maybe some very small number > 1)
(09:55:30 AM) sgallagh: So a limited set of COPR content that is useful
(09:55:47 AM) stickster: That's exactly what I understood.
(09:56:01 AM) sgallagh: Let me see if I can dig up recent information
(09:56:05 AM) stickster: phracek's PyCharm should be on that list I believe
(09:57:09 AM) sgallagh: stickster: https://fedoraproject.org/wiki/Changes/Playground_repository is the starting point
(09:57:51 AM) sgallagh: stickster: https://fedoraproject.org/wiki/Env_and_Stacks/Playground_repository_(draft) is a more technical look, but it seems unfinished.
(09:58:08 AM) sgallagh: Probably worth discussing with msuchy
(10:13:29 AM) jwb: halfline, do you know if anyone else has reported issues on f22 with the plymouth quit wait service never ending?
(10:13:49 AM) jwb: halfline, when i switch to a vt all i see is systemd saying it's waiting for that, and i get no login prompt
(10:15:41 AM) halfline: jwb: so that message is very misleading
(10:15:55 AM) halfline: it just means "waiting for boot to finish"
(10:16:08 AM) jwb: ok...
(10:16:46 AM) halfline: so it's probably unrelated to your issue. it either means boot never finished, or ... wait does it say "Starting" or "Started" ?
(10:17:46 AM) jwb: "A start job is running for Wait for Plymouth Boot Screen to Quit (1h 13min 40s/no limit)"
(10:19:37 AM) halfline: okay so something is blocking boot up from finishing
(10:19:39 AM) stickster: mclasen_: Are we able to provide the warning for users when enabling disabled repos on a *per-repo* basis?
(10:20:11 AM) halfline: jwb: if you hit ctrl-alt-f6 can you get to tty ?
(10:20:28 AM) halfline: hang on, gotta close polari it's jumping around every time i hit enter. let me open hexchat
(10:22:13 AM) halfline: ah much better
(10:22:41 AM) halfline: jwb: does your machine have any exotic video card?
(10:22:51 AM) halfline: like matrox g200
(10:23:00 AM) jwb: lol wut?
(10:23:01 AM) jwb: no
(10:23:07 AM) jwb: it's a macbook using i915
(10:23:26 AM) halfline: okay, so probably need journal output to know more
(10:23:37 AM) halfline: does ctrl-alt-f6 get you to a getty?
(10:23:37 AM) jwb: ctrl-alt-f6 gets me the thing i copied above
(10:23:51 AM) jwb: i get no login prompts on any of the vts
(10:24:19 AM) halfline: jwb: okay, can you reboot, and at the grub kernel command line put " 3 " ?
(10:24:36 AM) halfline: or systemd.unit=multi-user.target which is a lot more verbose way of saying the same thing
(10:24:43 AM) jwb: sure
(10:26:09 AM) jwb: halfline, ok, sitting at a login prompt now
(10:26:23 AM) halfline: okay log in as root
(10:26:28 AM) halfline: and type journalctl -b -a | fpaste
(10:26:49 AM) halfline: then open up /etc/gdm/custom.conf and add Enable=true to the [debug] section
(10:26:57 AM) halfline: reboot, reproduce
(10:27:07 AM) halfline: err
(10:27:14 AM) halfline: the journalctl line above is worng
(10:27:21 AM) halfline: journalctl -b -1 -a | fpaste
(10:28:00 AM) jwb: too large for fpaste. give me a second
(10:29:15 AM) mclasen_: stickster: you probably want to talk to hughsie, I'm sitting in all-day meetings
(10:29:21 AM) halfline: let me just enumerate since i confuzzled the instructions 1) log in as root 2) type journalctl -b -1 -a | fpaste 3) edit /etc/gdm/custom.conf and put Enable=true in [debug] 4) reboot 5) reproduce 6) reboot into 3 again 7) redo journal log
(10:29:35 AM) stickster: mclasen_: Ah, OK
(10:29:40 AM) halfline: change step 2 and 7 to whatever your plan is for them
(10:34:25 AM) jwb: halfline, https://jwboyer.fedorapeople.org/pub/journal.txt
(10:34:32 AM) jwb: halfline, https://jwboyer.fedorapeople.org/pub/journal-gdm-debug.txt
(10:34:38 AM) jwb: for steps 2 and 7 respectively
(10:40:29 AM) halfline: jwb: for step 7, did you see the same thing?
(10:40:37 AM) halfline: no graphics just frozen console ?
(10:42:07 AM) paragan: stickster, I see there are no f22 builds in https://copr-be.cloud.fedoraproject.org/results/phracek/PyCharm/
(10:42:15 AM) jwb: halfline, no, the graphics come up just fine. i can login via gdm if i wanted. i just never get a login prompt if i switch to a tty
(10:42:27 AM) halfline: OHHHH
(10:42:36 AM) halfline: jwb: gotcha
(10:42:41 AM) jwb: i know all about the kernel splats. working on those separately
(10:42:42 AM) halfline: jwb: does "sudo plymouth quit" fix it ?
(10:42:49 AM) jwb: let me reboot and try
(10:43:30 AM) halfline: it's probably https://bugzilla.gnome.org/show_bug.cgi?id=746498
(10:44:19 AM) ***halfline pushes that fix
(10:44:40 AM) jwb: halfline, well, that did something but not exactly what i wanted. might be hindered by the kernel issues there though since graphics decided to get all glitchy
(10:44:59 AM) halfline: jwb: let me just build the fix and you can try it
(10:45:08 AM) jwb: halfline, sure, that'd be great
(10:45:16 AM) ***jwb curses the i915 driver more than ever
(10:46:48 AM) sgallagh: stickster: I was just talking to phracek about the Playground repo
(10:47:11 AM) phracek: Hi sticker.
(10:47:14 AM) sgallagh: In addition to PyCharm, he also has a COPR for Google Android Studio that seems like a good fit for the Workstation target audience
(10:47:45 AM) phracek: Google Android Studio is not up to date now.
(10:47:59 AM) phracek: But Pycharm build for F22 is ongoing.
(10:48:31 AM) halfline: jwb: http://koji.fedoraproject.org/koji/taskinfo?taskID=9311000
(10:49:22 AM) phracek: If I understand correctly than COPR repositories will be included in Workstation but disabled, right?
(10:49:27 AM) phracek: stickster: ^
(10:49:46 AM) phracek: What kind of package going to be there?
(10:50:07 AM) jwb: halfline, thanks. will try it out
(10:50:07 AM) stickster: phracek: Not all of them. I started discussion about requesting inclusion of a small number in the Playground "repo of repos"
(10:50:25 AM) halfline: i guess i should do a 3.16.0.1 for that bug
(10:50:34 AM) sgallagh: phracek: Also "disabled" is a little fuzzy
(10:50:59 AM) phracek: sgallagh: Could you please explain it a bit for me?
(10:51:14 AM) sgallagh: They won't be installable blindly, but GNOME Software will be able to discover them and the user can install them with a click-through indicating that it's not "official" Fedora software.
(10:51:15 AM) phracek: sgallagh: Build for F22 is done. Pycharm is READY
(10:51:22 AM) sgallagh: \0/
(10:51:25 AM) ***sgallagh downloads
(10:51:49 AM) stickster: sgallagh: phracek: So there might really be two things to consider: (1) including Pycharm on its own; (2) including Pycharm in Playground
(10:52:20 AM) sgallagh: stickster: Explain what you mean, please?
(10:52:21 AM) stickster: sgallagh: AIUI it would be possible to do both without screwing up gnome-software or yum/dnf because the NVRs would be duplicate/identical
(10:52:58 AM) sgallagh: stickster: Take a step back and define what "including PyCharm on its own" means
(10:53:05 AM) sgallagh: Before we argue what can work :)
(10:53:13 AM) phracek: Pycharm in Playgroung is fine. The package has a problem with Packaging Guidelines and therefore it could not be in Fedora officialy
(10:53:21 AM) stickster: sgallagh: OK. What I mean is, a specific pycharm.repo, separately from playground.repo
(10:53:26 AM) stickster: sgallagh: Here's the thing. PyCharm repo, as phracek provides, is internally consistent, not likely to run into things like dependency problems, although that might happen in some Playground included COPRs
(10:53:55 AM) sgallagh: stickster: In this specific instance, it's a repo containing a single package
(10:54:11 AM) sgallagh: So that concern, while valid in general, is not valid in this specific example.
(10:54:12 AM) phracek: stickster: Specific pycharm.repo?
(10:54:13 AM) mclasen_: phracek: we probably need some appstream data or something in the copr to make the search work nicely
(10:54:23 AM) mclasen_: is hughsie in here ?
(10:54:24 AM) stickster: sgallagh: So in terms of warning the user, if we only solve the problem by including the PyCharm repo in the Playground, it would be covered by whatever warning we put on Playground... which will probably say "Some of this software may cause dep problems from time to time" (or s/th similar)
(10:54:40 AM) mclasen_: lets get hughsie in here before we continue
(10:54:46 AM) phracek: mclasen_: appstream data you mean metadata etc.
(10:54:46 AM) sgallagh: mclasen_: Seems sensible
(10:54:48 AM) stickster: sgallagh: Whereas that might scare off someone from using PyCharm even though it's unlikely to cause such a problem
(10:55:17 AM) phracek: mclasen_: appstream data can be added, though. It is easy.
(10:55:29 AM) sgallagh: stickster: I don't think we want to make that statement at all.
(10:55:31 AM) ***mclasen_ summons hughsie
(10:55:32 AM) stickster: mclasen_: +1, I asked hughsie in #fedora-devel but he hasn't fully answered question about per-repo text
(10:55:40 AM) sgallagh: The point of Playground is that it must remain curated so that such dep problems don't occur.
(10:55:55 AM) phracek: mclasen_: you are right, pycharm isn't mentioned in Gnome Software Center.
(10:55:58 AM) sgallagh: Unlike the wild west of COPR
(10:56:19 AM) mclasen_: hughsie: what do we need to add to the pycharm repo to make it show up in gnome-software ?
(10:56:23 AM) stickster: sgallagh: OK, that comes from my reading of "Problems" on the draft wiki page you pointed to above
(10:56:33 AM) hughsie: stickster, you can't do a per-repo enabled text right now; it would be tricky to localize etc
(10:56:55 AM) stickster: Hm. So whatever warning we put in has to work in all situations?
(10:57:01 AM) phracek: I have already did some work for Fedora packages and we need a pycharm.appdata.xml, right?
(10:57:21 AM) hughsie: stickster, there's a matrix of things; https://raw.githubusercontent.com/gnome-design-team/gnome-mockups/master/software/version2/wire-third-party-repo-dialogs.png
(10:57:50 AM) mirek-hm: sgallagh: I am here
(10:57:56 AM) jwb: halfline, yeah, that fixed it. awesome!
(10:58:04 AM) hughsie: mclasen_, what's the pycharm package name/
(10:58:05 AM) hughsie: ?
(10:58:21 AM) halfline: jwb: okay cool, i just did an upstream release with the fix too
(10:58:23 AM) ***mclasen_ passes the question to phracek
(10:58:51 AM) phracek: hughsie: Wait a minute.
(10:59:21 AM) phracek: hughsie: Name: pycharm-community
(10:59:33 AM) phracek: hughsie: Summary: Intelligent Python IDE
(10:59:49 AM) hughsie: phracek, i can't see anything in https://github.com/hughsie/createrepo_as_logs/tree/master/p
(10:59:57 AM) ***hughsie investigates
(10:59:58 AM) sgallagh: I need to run to the Server SIG meeting. I'll read the scrollback later
(11:00:11 AM) phracek: hughsie: Currently not it is only in Copr.
(11:00:32 AM) hughsie: phracek, so why would it shw up in gnome-software? :)
(11:00:34 AM) phracek: I didnt create a pycharm-community.appdata.xml file.
(11:01:04 AM) hughsie: Package not found: pycharm-community
(11:01:06 AM) phracek: hughsie: mclasen_ ask for that. 20 line above.
(11:01:27 AM) hughsie: ohh i see! it's only in the copr
(11:02:13 AM) phracek: hughsie: yeah. And if we plan to include it in Gnome Software Center then I will create XML file for PyCharm.
(11:02:15 AM) hughsie: phracek, you need to read https://blogs.gnome.org/hughsie/2015/01/09/finding-hidden-applications-with-gnome-software/ -- and then something needs to install the .repo file by default
(11:02:36 AM) hughsie: copr also needs to know hw to generate appstream metadata too
(11:02:49 AM) mclasen_: sure, we'll install the repo file, but what needs to be in the repository for search to work ?
(11:02:50 AM) hughsie: they were very keen at one sage, but it kinda got dropped on the floor
(11:02:58 AM) stickster: hughsie: mclasen_: presumably something like fedora-release-workstation (or a subpackage)
(11:03:26 AM) hughsie: stickster, i didn't get into the fesco policy thing, but something like that, yeah
(11:03:30 AM) hughsie: mclasen_, enabled_metadata=1
(11:03:47 AM) phracek: hughsie: I will test it. Thanks.
(11:03:50 AM) hughsie: and the repo needs to have appstream-metadata
(11:04:01 AM) hughsie: or i need to download it manually and add it to the fedora metadata
(11:04:09 AM) hughsie: (which migh tbe quicker to implement)
(11:04:25 AM) mclasen_: hughsie: enabled_metadata=1 is what goes in the .repo file
(11:04:33 AM) mclasen_: I asked what needs to go in the _repository_
(11:05:32 AM) stickster: mclasen_: Right, it's not clear in the hughsie blog post where the metadata should be (specific name? location?)
(11:06:11 AM) hughsie: stickster, so we have two choices here
(11:06:30 AM) hughsie: one is that the repo manager does something like appstream-builder on the repo
(11:07:02 AM) hughsie: something like https://blogs.gnome.org/hughsie/2014/12/17/actually-shipping-appstream-metadata-in-the-repodata/
(11:07:28 AM) hughsie: the other cheat is that i could just dwnload the copr and regenerate it into the fedora metadata like i do for fedora
(11:07:29 AM) kalev: as a side thought, I wonder if we could get the copr people to automatically run appstream-builder on all repos
(11:07:45 AM) hughsie: kalev, the sticky issue is we have to download screenshots form upstream
(11:07:52 AM) hughsie: so random network access
(11:08:12 AM) kalev: ahh right, yes
(11:08:18 AM) mclasen_: hughsie: whats the repo manager ? a tool ? a person ?
(11:08:34 AM) mirek-hm: kalev: I can do that but the programs which generate that needs to be in fedora, copr-be currently run on F20 IIRC
(11:08:42 AM) hughsie: mclasen_, could be either
(11:09:02 AM) kalev: mirek-hm: they should all be in fedora
(11:09:14 AM) hughsie: kalev, i think he means a new enough appstream-builder
(11:09:35 AM) hughsie: mirek-hm, if it's just a few coprs that i can rsync from it would be easy to add to my daily rebuilds
(11:10:08 AM) hughsie: e.g. https://github.com/hughsie/appstream-scripts/blob/master/fedora/fedora-22.sh
(11:10:28 AM) mirek-hm: kalev: https://bugzilla.redhat.com/show_bug.cgi?id=1105587#c4
(11:10:41 AM) ***hughsie has a 2 year old hanging from their neck.. :)
(11:10:42 AM) mclasen_: I'd like to see a solution for pycharm and f22, before we get all distracted by looking a coprs in general
(11:11:33 AM) kalev: the quick solution is to ship pycharm metadata in the appstream-data package in fedora proper
(11:11:41 AM) hughsie: right
(11:11:47 AM) hughsie: pycharm would need appdata
(11:12:22 AM) mclasen_: well, thats not a solution, really
(11:12:37 AM) hughsie: phracek, so we certainly need an appdata file as a first step
(11:12:37 AM) phracek: hughsie: I will create appdata for PyCharm tomorrow. Just for testing.
(11:12:51 AM) hughsie: we also need to work out how to mirror the upstream screenshots in a copr
(11:13:05 AM) hughsie: i.e. usint alt.fedora
(11:13:10 AM) phracek: hughsie: I'll inform you.
(11:13:46 AM) hughsie: same for chromium i guess
(11:14:48 AM) hughsie: i guess it's really an integration issue for the copr guys
(11:15:01 AM) kalev: mirek-hm: I think I saw nirik mention somewhere that it would be trivial to upgrade copr builders to F21
(11:15:07 AM) mclasen_: so gnome-software will find the appdata if it is in the copr repository ?
(11:15:27 AM) mirek-hm: kalev: builders are already on f21, backend and frontend are f20
(11:15:29 AM) hughsie: yup, if its in repomd.xml then it gets autodownloaded
(11:15:30 AM) nirik: kalev: I might even say this has already been done.
(11:15:42 AM) kalev: ahh :)
(11:15:58 AM) hughsie: the main issue for them might be the random network access required
(11:16:31 AM) hughsie: maybe we do need to jam the screenshots into a subpackage or something
(11:16:40 AM) mirek-hm: hughsie: I can do it, there are few bumpers, but it is doable, I will give it some priority
(11:16:40 AM) phracek left the room (quit: Quit: Leaving).
(11:16:43 AM) hughsie: would make me hugely unpopular with the mirror admins
(11:17:26 AM) mirek-hm: hughsie: we are at the end of migration to new fedora-cloud, and I expect that after 10 days I can start on this.
(11:17:37 AM) hughsie: awesome
(11:17:44 AM) kalev: that's really cool, indeed
(11:17:47 AM) hughsie: yell if you want a call to speed up niggles
(11:18:35 AM) hughsie: nirik, this is what i use right now: https://github.com/hughsie/appstream-scripts/blob/master/fedora/fedora-22.sh
(11:19:18 AM) ***nirik is in another meeting right now, can't really follow here.
(11:23:13 AM) ***stickster trying to understand what the mirror-admin issue is for those screenshots...
(11:23:23 AM) stickster: too much shorthand makes for limited understanding :-(
(11:23:48 AM) hughsie: stickster, we download the screenshots from upstream and mirror them to alt.fedora
(11:23:53 AM) hughsie: (privacy, and scale)
(11:25:10 AM) mirek-hm: hughsie: what is alt.fedora?
(11:25:25 AM) stickster: So the problem there is a more frequent delta for downstream mirrors?
(11:25:27 AM) kalev: mirek-hm: https://alt.fedoraproject.org/pub/alt/screenshots/
(11:27:45 AM) hughsie: stickster, well, what we want to avoid is putting links to upstream screenshots in the distro metadata
(11:27:58 AM) hughsie: stickster, some upstreams would fail with the additonal load
(11:28:18 AM) hughsie: some people see it a privacy issue if upstream can tell if you're looking at their software
(11:28:30 AM) hughsie: e.g. by ip
(11:28:33 AM) stickster: *nod
(11:28:48 AM) hughsie: so we copy them to alt.fedora
(11:28:56 AM) hughsie: and resize them to the right sizes
(11:29:26 AM) hughsie: so we only download a tiny thumbnail for the screenshot thumbnails, rather than a 1024x768px just for a 124x preview
(11:29:42 AM) hughsie: in an ideal world
(11:29:59 AM) hughsie: when building a package in the copr
(11:30:04 AM) hughsie: we'd rerun the builder tool
(11:30:16 AM) hughsie: and re-download the upstream screenshot if it's different
(11:30:24 AM) hughsie: and then save to somewhere copr.fedora
(11:30:37 AM) hughsie: (the builder tool deals with resizing and rewriting URLs)
(11:31:37 AM) stickster: Right, so that's the long-term solution, if possible... make that happen within copr repo generation
(11:31:43 AM) hughsie: right
(11:31:57 AM) hughsie: and if they can run appstream-builder it pops out an appstream file
(11:32:12 AM) hughsie: which you can trivially add to repomd.xml using modifyrepo
(11:32:31 AM) hughsie: which means it gets automatically downloaded if enabled_metadata=1 is set in the repo file
(11:32:42 AM) hughsie: there are a lot of "if's" there i know
(11:33:34 AM) hughsie: the alternative is we define a "sane" set of coprs we want
(11:33:42 AM) hughsie: and i mirror them on my server
(11:34:02 AM) hughsie: and we update the appstream-data package monthly
(11:34:15 AM) stickster: hughsie: yup, that's what mclasen_ was getting at... for F22 we're likely stuck with that solution
(11:34:18 AM) hughsie: although that's kinda sucky
(11:34:22 AM) stickster: agreed
(11:34:37 AM) hughsie: and in a "hit by a bus" situation isn't awesome either
(11:34:50 AM) hughsie: although, great for my job security :)
(11:35:09 AM) mclasen_: no, I don't think we want to munge copr into the builtin appdata
(11:35:19 AM) hughsie: it gets kinda messy
(11:35:30 AM) mclasen_: I'd rather try the modifyrepo thing with the one or two coprs we target now
(11:35:41 AM) hughsie: +1
(11:35:52 AM) hughsie: if the coprs are on f21 it should be pretty easy
(11:35:56 AM) ***stickster wonders whether hughsie, florian, dgilmore, et al. ever met as follow-on to our Brno meetup as planned
(11:36:29 AM) hughsie: stickster, not really; there are still big philosophical disagreements there
(11:37:12 AM) stickster: yeah, because this comes back to whether we can do something as e.g. a koji plugin, which I think ppl are still dancing around
(11:37:42 AM) hughsie: stickster, there's policy too, like for instance the fedora builders can't access the internet
(11:38:24 AM) stickster: alternative is (I guess?) packager pain
(11:38:27 AM) hughsie: and jamming a few tens of gigs of screenshots into each fedora release isn't going to make me popular
(11:38:36 AM) stickster: right
(11:38:39 AM) hughsie: stickster, we could automate quite a lot
(11:38:46 AM) hughsie: but it's a lot of large images
(11:39:05 AM) hughsie: for something like libreoffice it's several screenshots per app
(11:39:21 AM) hughsie: so a src rpm for libroffice would be a *lot* bugger
(11:39:25 AM) hughsie: *bigger!
(11:39:51 AM) mclasen_: well, a few megabytes hardly matter when we talk about libreoffic
(11:39:52 AM) mclasen_: e
(11:40:37 AM) hughsie: mclasen_, the alternative is we require a lot of packagers to upload a ton of screenshots
(11:40:49 AM) hughsie: and keep track when they change :/
(11:45:33 AM) mclasen_: so, what are the concrete steps we need to talk to make pycharm searchable ? build appstream metadata and call modifyrepo at some point ? can copr repos be modified like that ?
(11:49:56 AM) mirek-hm: mclasen_: speaking as copr guy - I do not know (exactly), it can be done, however I will not be doing that for one copr. When I will be doing it, it will be general solution for all coprs
(11:50:24 AM) stickster: mclasen_: I think phracek said he was amenable to do this for his repo, assuming we want to include it (which I think we do)
(11:50:56 AM) mclasen_: mirek-hm: will we have that 'general solution' for f22 ?
(11:51:03 AM) stickster: mclasen_: i.e. He would build/include the appdata needed, run modifyrepo. We would need to provide the .repo file somewhere appropriate
(11:51:20 AM) mirek-hm: when exactly is GA of F22?
(11:51:25 AM) hughsie: stickster, fedira-release-worksatiob seems best
(11:51:25 AM) mclasen_: stickster: the thing is that I am not sure if you can modifyrepo on a hosted copr repo ?
(11:51:34 AM) stickster: mirek-hm: 2015-May-19 IIRC
(11:51:38 AM) hughsie: mclasen_, sure, why not?
(11:51:48 AM) stickster: hughsie: I see you hired my typist :-)
(11:52:13 AM) hughsie: stickster, one handed :/
(11:52:28 AM) mirek-hm: mclasen_: I think it is doable
(11:52:31 AM) hughsie: one hand trying convince 2yo not to "help"
(11:52:49 AM) hughsie: ill be back to top form in 2h
(11:53:28 AM) stickster: np, I was just joking :-)
(11:53:40 AM) mclasen_: hughsie: I just don't know how it works - don't you have to have shell access to the hosting machine to modify the repo ?
(11:54:06 AM) mirek-hm: you could not modify repo data on copr, definitely
(11:54:57 AM) hughsie: mirek-hm, you do createrepo, right?
(11:55:13 AM) mirek-hm: createrepo_c to be precise
(11:55:56 AM) hughsie: right, spo just run modifyrepo_c after that
(11:56:00 AM) mirek-hm: I meant normal user could not modify it. We - as fedora infrastrucure - can, of course
(11:56:09 AM) hughsie: right
(11:56:23 AM) stickster: ttps://git.fedorahosted.org/cgit/copr.git/tree/backend/backend/createrepo.py
(11:56:26 AM) stickster: oops
(11:56:28 AM) stickster: https://git.fedorahosted.org/cgit/copr.git/tree/backend/backend/createrepo.py
(11:57:01 AM) hughsie: maybe it does make sense to embed the screenshot data in the package for copr packages
(11:57:25 AM) hughsie: unless the copr infra can access the net
(11:57:31 AM) nirik: ok, reading back, it sounds like this will all get taken care of in the copr? or is there anything I need to look at? :)
(11:57:38 AM) nirik: it can I think
(11:58:48 AM) hughsie: nirik, so https://blogs.gnome.org/hughsie/2014/12/17/actually-shipping-appstream-metadata-in-the-repodata/ should work
(11:59:54 AM) ***nirik reads
(12:01:54 PM) kalev: hughsie: if I remember it right, one thing that dgilmore said in Prague or whereever the fedora conference was,
(12:02:15 PM) kalev: hughsie: was that screenshots should ideally be available during package build time, so that it's possible to validate them
(12:02:27 PM) kalev: hughsie: or error out if they are missing or whatever
(12:02:44 PM) hughsie: kalev, that's an aweful lot of screenshots in the srpms
(12:02:49 PM) kalev: but I've forgotten where he wanted to store them
(12:02:51 PM) kalev: yeah
(12:03:17 PM) hughsie: and wed have to upload them at fedpkg time
(12:03:32 PM) nirik: I think thats good progress. ;) perhaps we can make something work...
(12:03:34 PM) hughsie: i.e. in the lookaside
(12:03:37 PM) kalev: but I don't think the srpm size really matters, the srpms are already huge and a few gigabytes more to the whole collection wouldn't make that much of a difference
(12:03:56 PM) hughsie: kalev, *5 sizes
(12:04:16 PM) kalev: what I'm trying to say that we're a binary distro
(12:04:26 PM) kalev: and the things that end users download are binary rpms, not srpms
(12:04:39 PM) kalev: and even if the srpm size explodes, it's rarely downloaded by the users so it doesn't matter that much
(12:04:40 PM) hughsie: kalev, we'd also need a ton of -screenshots subpacages
(12:05:00 PM) hughsie: and also we'd need to mirror them
(12:05:55 PM) kalev: no, I mean more like a scheme where during a build, the screenshots are copied from the srpm to somewhere in the build host
(12:06:08 PM) kalev: as in, they wouldn't be _inside_ the binary rpms, but next to them
(12:06:23 PM) kalev: and then as a separate step, they'd be collected and copied to a web server
(12:06:56 PM) hughsie: kalev, how do we get them in the srpm?
(12:07:22 PM) kalev: I don't know
(12:07:47 PM) hughsie: they'd have to be in the lookaside
(12:08:00 PM) hughsie: and then we're storing them in *3* splaces
(12:08:01 PM) kalev: and putting them there is a lot of manual work and prone to error
(12:08:16 PM) hughsie: and we'd need to upload them using fedpk-new-sources
(12:08:24 PM) hughsie: i.e. not automatically
(12:08:37 PM) hughsie: right
(12:09:34 PM) kalev: hughsie: do you know how DimStar solved the screenshot problem?
(12:10:10 PM) kalev: I saw him talk about this, but I'm afraid I totally didn't listen :)
(12:11:22 PM) hughsie: biab
(12:14:42 PM) hughsie: kalev, he hasn't yet
(12:14:51 PM) hughsie: atm they're downloading from the net
(12:14:55 PM) mclasen_ left the room (quit: Quit: mclasen_).
(12:14:58 PM) kalev: ah
(12:15:02 PM) hughsie: but he wants appstream-builder to chuck out a list of URLs
(12:15:15 PM) hughsie: which someone can run on a machine with network access
(12:15:16 PM) mclasen_ [mclasen@nat/redhat/x-ibzcccqnnxkoffxv] entered the room.
(12:15:23 PM) hughsie: which puts them in the right place on alt.
(12:15:57 PM) hughsie: i'm not totally convinced
(12:16:13 PM) hughsie: (e.g. if the upstream URL isn't found, we;d have to have a fake screenshot)
(12:16:22 PM) hughsie: but it's probably a compromise
(12:17:07 PM) kalev: I wonder if putting the screenshot urls in the appdata was a good idea, in retrospect
(12:17:24 PM) kalev: should have maybe had all the upstream package install them somewhere in /usr
(12:17:25 PM) hughsie: dude, it's not like we can't change the spec :)
(12:17:44 PM) hughsie: kalev, i think a lot of upstreams would have been upset by that
(12:18:14 PM) hughsie: and there'd be a lot of --disable-screenshots required
(12:18:29 PM) kalev: indeed
(12:18:35 PM) hughsie: kalev, but i'd be up for supporting that if you think it's a better solution
(12:19:00 PM) hughsie: i'd have a hard time writing a blog that told people to install screenshots
(12:19:12 PM) mclasen_: if you put the screenshots in the package, you can't see them before installing the package...
(12:19:22 PM) hughsie: a lot of upstreams don't put the screenshots in vcs for instance
(12:19:25 PM) kalev: yes, of course
(12:19:48 PM) kalev: mclasen_: but we could extract them _from the packages_ and upload to the web server, during repo compose time
(12:19:58 PM) kalev: as opposed to downloading them from the net
(12:20:14 PM) mclasen_: I'm sure nirik has a few choice comments about that idea
(12:20:16 PM) kalev: the downloading part is the hard problem here that has made it all very difficult
(12:20:34 PM) hughsie: kalev, we could implement dimstar's idea
(12:21:07 PM) kalev: hughsie: yep, that could work -- how would you do downstream overrides for screenshots, in that sceme?
(12:21:10 PM) kalev: scheme
(12:21:36 PM) kalev: patch appdata and point to a different screenshot url?
(12:21:46 PM) hughsie: kalev, i'm not sure yet; that's probably the most sane
(12:21:51 PM) ***kalev nods.
(12:21:59 PM) nirik: downloading stuff is bad. It means you can't have reproducability... granted they are just screenshots, but if site X goes away, suddenly you have no screenshot, or it gets replaced by a junk one.
(12:22:00 PM) hughsie: less magic than what we have now
(12:22:05 PM) hughsie: but urgh, a lot of patches
(12:22:34 PM) hughsie: nirik, the alternative is we have huge tarballs and srpms
(12:22:55 PM) hughsie: and have to convince upstreams to install extra files by default :/
(12:23:03 PM) hughsie: one xml file is hard enough
(12:23:23 PM) mirek-hm: I suppose we can treat those urls like url in Source0: it is nice to know where is upstream of that image, but we will take just the basename and store it somewhere
(12:23:55 PM) hughsie: mirek-hm, that's making the packagers do an aweful lot manually
(12:24:04 PM) nirik: well, if we cared about space we wouldn't be shipping 0ad or webkitgtk4 debuginfo. ;)
(12:24:09 PM) nirik: or 35000 packages
(12:24:11 PM) hughsie: i really want any packaging stuff to be done automatically
(12:25:23 PM) hughsie: sooner or later you have to get a random picture from upstream
(12:25:40 PM) hughsie: and i can't see projects agreeing to put them in the tarball and installing them by default
(12:25:57 PM) kalev: I can see GNOME agreeing to that :)
(12:26:05 PM) hughsie: dimstar's idea might be the best compromise
(12:26:18 PM) hughsie: kalev, sure, perhaps 20 maintainers
(12:26:27 PM) hughsie: kalev, there are a lot more apps than just GNOME tho
(12:27:02 PM) mirek-hm: hughsie: no, I not meant to put is as SourceX in spec, just to treat it with the same logic. rpmbuild does not download that tar.gz in Source0 despite the fact that there is url
(12:27:30 PM) hughsie: mirek-hm, yes, but it does seem like we're trying to push a square peg in a round hole
(12:27:57 PM) hughsie: of course, the user could just download the image direct from upstream
(12:28:10 PM) hughsie: although that means we have to download them at full source resolution and also privacy
(12:30:32 PM) kalev: hughsie: I wonder if it would help to have screenshot checksums in appdata
(12:30:47 PM) hughsie: kalev, what would that help with?
(12:31:00 PM) hughsie: to prevent MITM attaacks?
(12:31:05 PM) kalev: so that if something on the fedora infrastructure side downloads them, they could verify the screenshots against the metadata
(12:31:08 PM) kalev: yep
(12:31:16 PM) mclasen_: attacked by a screenshot ?
(12:31:22 PM) ***mclasen_ can see the headlines
(12:31:24 PM) kalev: :)
(12:31:31 PM) hughsie: we can do checksums for releases, so it woudn't be hard to add
(12:32:07 PM) hughsie: kalev, easiest would just be to allow copr builder to access the internet
(12:32:31 PM) hughsie: because even if we do a file of URLs to download, we still need to download them
(12:32:48 PM) hughsie: unless it's me sitting on my microserver, but urgh
(12:33:26 PM) hughsie: kalev, i wonder if we're missing a trick
(12:33:38 PM) mclasen_: can we do a script that unpacks the srpm, sniffs out the appdata, downloads the screenshots, and adds them as extra sources ?
(12:33:41 PM) kalev: I guess it could work to give the builder host access to the net, but disable net access from within mock
(12:33:46 PM) mclasen_: and generates a new srpm ?
(12:34:29 PM) hughsie: mclasen_, sure, that would work too, but that script would need to be run on a server with network access, and then you might as well do what kalev is suggesting
(12:34:33 PM) kalev: so that srpm -> binary rpm has no net access, but when the repo compose happens, the repo compose host opens up the rpms, extracts appdata, downloads urls, and writes them out
(12:35:06 PM) hughsie: kalev, kinda what my microserver has been doing for a few years :)
(12:35:11 PM) kalev: :)
(12:35:24 PM) kalev: nirik: is ^^ something that might be conceivable in the fedora build system?
(12:35:29 PM) mclasen_: hughsie: yes, I am talking about some srpm preprocessing
(12:35:39 PM) mclasen_: so you can feed the srpm to a builder without network access
(12:36:12 PM) mclasen_: this whole thing is just too complicated
(12:36:16 PM) hughsie: the builder doesn't really need network access; like kalev said it's the compose step that does
(12:36:30 PM) mclasen_: rel-eng is not going to accept it
(12:36:39 PM) mclasen_: we've banged our head against that wall for a while...
(12:36:46 PM) hughsie: mclasen_, other ideas welcome :/
(12:36:50 PM) mclasen_: time to stop doing it
(12:36:55 PM) mclasen_: I gave you one 5 lines up
(12:37:14 PM) hughsie: mclasen_, so the user runs the srpm->srpm tool when they update the package?
(12:37:23 PM) mclasen_: what ? no
(12:37:27 PM) mclasen_: the user never sees srpms
(12:37:42 PM) mclasen_: the person stuffing the srpm into a builder will have to run it
(12:37:47 PM) nirik: right, builders don't have any net... but compose time it could perhaps. I think already some things like comps git and such are gotten...
(12:38:04 PM) hughsie: mclasen_, you've just moved the problem from compose->builder
(12:38:06 PM) mclasen_: of course, it should also be possible to manually write an srpm that has the screenshots included
(12:38:28 PM) mclasen_: hughsie: neither the builder nor the compose machine have net access
(12:38:54 PM) mclasen_: all input needs to come in via the official inputs ie the srpm / spec file+ tarballs
(12:39:01 PM) hughsie: mclasen_, so where does your script run
(12:39:04 PM) hughsie: sorry if i'm being dense
(12:39:17 PM) mclasen_: on your system, if you have a copr that you want to put an app in
(12:39:42 PM) mclasen_: basically, I'm telling you to download your screenshots, wrap them in a tarball, add a Source2: line to your spec
(12:39:51 PM) hughsie: ahh sorry, so this is just for the copr where you upload a srpm
(12:39:51 PM) paragan left the room (quit: Quit: Leaving).
(12:40:13 PM) mclasen_: I guess the lookaside problem is a bit different
(12:40:31 PM) mclasen_: since the builder is not only unable to download stuff, but also unable to upload stuff
(12:40:39 PM) hughsie: right
(12:40:50 PM) kalev: the builder is unable, but the compose host has more access
(12:41:04 PM) kalev: the compose host can definitely _upload_ stuff to fedora infrastructure
(12:41:13 PM) kalev: how else do the rpms get out of koji?
(12:41:19 PM) hughsie: maybe putting the screenshots in the .tarball would have been more sane
(12:41:31 PM) hughsie: although i think a lot of people would have issues with that
(12:42:48 PM) hughsie: the alternative is we have a central app-store model when people submit stuff online
(12:42:50 PM) mclasen_: maybe we should just ditch the whole lookaside idea
(12:43:04 PM) mclasen_: its a solution for a problem we haven't actually seen happening
(12:43:18 PM) hughsie: well
(12:43:21 PM) mclasen_: people happily put their apps' website in the about dialog
(12:43:25 PM) hughsie: privacy people kinda care
(12:43:28 PM) kalev: well, it's also caching the thumbnails
(12:43:37 PM) mclasen_: and we don't download that and put it in a lookaside
(12:43:38 PM) hughsie: some source images are HUGE
(12:43:50 PM) kalev: right now upstream just has one screenshot url
(12:43:56 PM) mclasen_: privacy people can have a 'no screenshots please' toggle
(12:43:57 PM) hughsie: mclasen_, sure we pre-resize the images when uploading to alt.fedora
(12:44:02 PM) hughsie: mclasen_, true, true
(12:44:15 PM) kalev: and hughsie's appstream builder makes a low resolution copy of it for initial showing
(12:44:29 PM) kalev: so that the gnome-software details page can instantly show thumbnails
(12:44:34 PM) kalev: and load the big screenshot only when clicked
(12:44:40 PM) mclasen_: thats all nice and stuff... but if we can't solve the build infrastructure issues, its all moot
(12:45:08 PM) hughsie: kalev, maybe if appstream-builder is run with --nonet we just don't munge the screenshot url at all
(12:45:22 PM) hughsie: i.e. for copr we don't care about the mirroring
(12:45:40 PM) hughsie: and we don't care about the blurred loading / delay
(12:45:42 PM) kalev: hughsie: oh yes, maybe we can just ignore the whole screenshot problem for now, just to get something out
(12:46:05 PM) kalev: if we can get metadata shipped in the repo, that would already be a huge step forward
(12:46:11 PM) hughsie: for copr, right
(12:46:14 PM) kalev: and can work on the screenshot issue separately, at a later time
(12:46:23 PM) kalev: yes, I don't think it would work for a whole of fedora :)
(12:46:44 PM) hughsie: kalev, can you open a bug against appstream-glib about no munging for --nonet and i'll fix up tonight
(12:46:52 PM) kalev: sure
(12:47:53 PM) ***hughsie has to make dinner in a few mins
(12:47:55 PM) mclasen_: hughsie: here's another trick idea: do the lookaside population on demand
(12:48:09 PM) mclasen_: instead of downloading the url when gnome-software needs it, have it talk to some web service
(12:48:19 PM) mclasen_: that then gets it and caches it
(12:48:21 PM) kalev: ohh, like a caching proxy?
(12:48:26 PM) mclasen_: yeah
(12:48:57 PM) mclasen_: that cs for you... solve all your problems with another level of indirection
(12:49:04 PM) kalev: that might just be the best idea so far
(12:49:42 PM) hughsie: mclasen_, sure, i'd be up for that, but i'm not going to be writing a webservice anytime soon
(12:50:34 PM) mclasen_: nirik: is that something we could host on fedora infrastructure ?
(12:51:34 PM) ***nirik isn't fully clear what you want there...
(12:51:40 PM) nirik: whats requesting the url?
(12:53:36 PM) hughsie: i suppose gnome-software
(12:54:05 PM) kalev: nirik: the scenario is that gnome-software wants to show a screenshot of an app
(12:54:05 PM) hughsie: fedoraproject.org/mythical_app.py&size=102&id=gnome-terminal.desktop
(12:54:18 PM) kalev: nirik: and it knows the url of the screenshot
(12:54:48 PM) kalev: nirik: but instead of going and downloading the screenshot directly, it asks a fedora proxy server to relay the image
(12:55:05 PM) kalev: nirik: and also possibly resize it too
(12:55:20 PM) nirik: is this urgent? i am getting hit with about 50 people pinging me today and can't concentrate on anything. Starting to get cranky.
(12:55:32 PM) kalev: not urgent, sorry :)
(12:55:33 PM) hughsie: the proxy would need random internet access i guess
(12:55:50 PM) nirik: not your fault, I just can't think with all the people bothering me. ;)
(12:55:58 PM) hughsie: although it does take this out of the build process
(12:56:59 PM) hughsie: kalev, the problem then becomes "allow the proxy to have random access to the internet"
(12:57:11 PM) hughsie: which i think would be just as unpalatable
(01:04:14 PM) hughsie is now known as hughsie-afk-cook
(01:04:43 PM) kalev: hughsie-afk-cook: https://github.com/hughsie/appstream-glib/issues/43
(01:05:05 PM) mclasen_: hughsie: why would that be unpalatable ?
(01:05:49 PM) mclasen_: I guess you don't want to allow caching random content
(01:05:57 PM) kalev: I think the "no net access" rule only applies to the build system and not to some random proxy service
(02:16:53 PM) kalev is now known as kalev-afk
(03:25:14 PM) kalev-afk is now known as kalev
(03:28:42 PM) hughsie-afk-cook: kalev, fixed that appstream issue
(03:29:10 PM) kalev: hughsie-afk-cook: awesome!
(03:31:59 PM) mirek-hm is now known as mirek-hm-zzz
(03:33:30 PM) mclasen_: kalev, hughsie-afk-cook: now that 3.16. is out, it would be cool to merge rluzinskis shrinking stuff
(03:34:20 PM) hughsie-afk-cook: mclasen_, remind me, srinking?
(03:34:57 PM) mclasen_: there's a patch in bugzilla that makes the recommended section hide tiles if the screen is too narrow
(03:35:06 PM) mclasen_: the '1024x768' issue
(03:35:52 PM) kalev: hughsie-afk-cook: https://bugzilla.gnome.org/show_bug.cgi?id=742211#c13
(03:36:17 PM) kalev: I think it's really cool :)
(03:36:33 PM) hughsie-afk-cook: talking of which, do you want me to branch all my stuff?
(03:39:25 PM) kalev: aday__: have you seen the screencast in https://bugzilla.gnome.org/show_bug.cgi?id=742211#c13 ?
(03:40:19 PM) hughsie-afk-cook: kalev, okay, i've branched, feel free to start the insanity
(03:40:46 PM) kalev: hughsie-afk-cook: great!
(03:41:05 PM) hughsie-afk-cook: mclasen_, i also wrote up https://blogs.gnome.org/hughsie/2015/03/24/how-to-turn-the-chromebook-pixel-into-a-proper-developer-laptop/
(03:41:14 PM) hughsie-afk-cook: alex asked me to do something for the other GNOME laptops
(03:46:45 PM) hughsie-afk-cook: back in a bit
(03:46:46 PM) hughsie-afk-cook left the room (quit: Quit: Leaving).
(04:14:31 PM) The account has disconnected and you are no longer in this chat. You will automatically rejoin the chat when the account reconnects.