hi i’m brian exobeard bax i am the red hat liaison to the centos board and so as such i actually speak on behalf of red hat when i speak for the board or speak at the board meetings and so i’m the only member of the board who does not represent themselves i work for the rel business unit and you’ll learn more about me in the next talk

thomas hi i’m tobas i’m working at cern i joined the board last year i’m interested in everything around distribution so go ahead ask questions

david hi i’m davide i’m a production engineer at facebook i within santos i chair the hyperscale sig i joined the board this year and happy to answer any questions you might have

hey josh boyer newly appointed board member with davide uh i work on rel in the linux engineering organization

everybody goes through that right yeah i hope so right so my name is johnny hughes and i’ve been building and signing centos packages since about 2003 um so if you guys have any questions let me know and i work oh i work in internally inside of red hat and i am on the uh cpe team the team that actually takes care part of our job is to take care of the infrastructure for centos as well as uh the infrastructure and and the startup of centos stream 89

and pad hi i’m pat i joined the board about the same time as tomas i’ve been working in the rebuild space for uh over a decade both uh informally with the centos community and uh directly on scientific linux uh work with fedora on a couple things here and there i am always happy to chat with you about free software projects so as you’ve heard we have a wide range of expertise here please do ask your questions

all right i will i will start then with a question that i was asked on irc this morning it’s already been answered on the mailing list but uh when will centos stream nine be available do you want me to cover that one i guess it depends on the definition are yeah that’s exactly where i was gonna go with it um so we have centos stream nine uh production composers in place uh you can find them they are going to be going to the mirrors very soon i believe i think we actually have it on the centos mirror network but we don’t have um like a centos repos definition in place i know the team was working on that either last week or this week but any day now in terms of normal availability that most people would think of

yeah on the stream nine front uh do we have uh our release notes or some documentation for things that we’re excited about up that we can point people towards no not that i know that’s a good question though um i’m not aware of us producing release notes for stream uh i would say at the point when real nine beta is released we can certainly leverage those but stream will have already you know moved on towards what we’re doing to ga shouldn’t be that much different though considering it’s a major release there was some talk of making the documentation public in a git repo somewhere so that folks could contribute to it before it was out varelle i don’t know what’s the stage of that uh i said it was a triad on centos docs on this as well yeah i actually don’t know the answer to that question either uh carl is is uh pointing out that we do have a centaur stream release package that has centos stream repos in it and brian says we’re still working with the docs team

i’ll give a personal answer to that is i’ve been enjoying toolbox on fedora to be able to manage my containers and it will be available on el nine this will be very exciting to to be able to run containers um easily with graphic applications and things so if you didn’t check out this project yet i encourage you to to have a look to to toolbox it’s very very nice project and it helps a lot if you’re desktop user and want to complete the rise applications

we have two questions in the chat and i’m going to read this aloud for the purpose of the recording the first one is what’s going to replace sig core t functional for cs9 in particular i’m interested in compose mirror content gating will we be able to see test status somewhere and contribute tests to it i’m also interested in this if anybody knows the answer we should have invited brian to join this panel um his answer for the recording is he suggests focus on package level testing uh through git lab right now so i don’t know that we have an answer to the what is going to replace it um but that is suggested approach is the functional actually going away i thought the plan was to keep it around and have something else but i’m not sure to be honest

we have a question from luna who says i saw rel 8.5 beta was released a couple of days ago will sent a stream eight update to the stable version of that rel version or is it nine only is it nine that only will be supported in the future can i can i take this one really quickly as a user point of view rather than someone working at red hat so we had an interesting thing is we had to debug a problem with some intel cpus and basically centos stream is the upstream of rail 8.5 so the kernel that you have in stream now is the canon that would be shipped with 8.5 so we could test basically uh our intel cpu with stream and it was working well it was not working with 8.4 and so we are sure that it’d be we’re working in 8.5 so you have to remember that stream is already if you want to rail 8.5 at this stage is it clear and maybe some people want to add on that yeah uh i’d add to that uh stream uh eight is running to 2024. uh the the support for stream eight isn’t going anywhere uh stream nine is where the exciting things are kind of happening for a lot of us here on the board because this is a new and exciting and lively project but stream mate is going to hang around until 2024. um it’s it’s not going anywhere so don’t don’t worry about that disappearing on you that is something that we feel very very important streamlight is going to be here 2024. and as far as centos linux 8 is concerned the downstream build uh we are going to build centos linux 8.5 uh off based off of rail 8.5 that’s going to get built it’s going to get released and then depending on whether that happens before or after the end of this year uh it’ll either be available for a long period of time uh you know if i don’t know when 8.5 braille 8.5 is going to come out but uh at december 31st um that’s the eol for centos linux eight not centos stream eight which is as pat said is going to be around for until 2024. uh centos linux 8 will still get moved into the vault area so vault.syntax.org we’ll have the final release version of centos linux 8.5 and all of the zero day updates that come out as part of of uh the source code for for rel 8.5 all of those updates that are zero day updates and all of the ga release for 8.5 will go out they will it will be on vaultlessness.org and then it won’t get any more updates after that so basically 8.5 on ga day uh and any updates that actually physically happen before december 31st of this year will get rolled in to what we have is 8.5 or centos 8 linux and nothing after that besides the zero day updates if the for some reason the rel 8.5 gets delayed so that that’s what will be in the vault now we don’t recommend that you actually use that in production after updates stop happening because there’s going to be security issues in that you know in that 8.5 area once red hat is producing uh red linux 8 updates for security that we’re no longer rolling into a downstream centos linux 8. so we don’t recommend that you use that in fact you should you probably shouldn’t be using it right now it’s only two months until it goes away if you haven’t already made plans to move off of centos 8 linux you’re way behind so you need to be doing something to to plan on the fact that it’s not going to be here uh after december

we’ve got a few more questions um first we have an anonymous question what is the state of modularity incentive stream nine i i can take that one um will continue to exist in stream nine and real nine uh the usage of it is a little bit reduced in scope based on what you saw in rail eight perhaps i think in stream nine composes today we have a one or two modules both related to our container tooling i actually expect that to go away before row 9ga because we’re doing some repackaging of how we’re doing those tools but then we’ll have modules introduced in later versions of a rel and they’ll materialize and stream before they show up there Laughter well it’s a start that’s right uh brian points out and this is actually really something that’s good to highlight uh we were able to actually do this modular development and stream and and get it in there and prove that it’s actually working including a slight change in the module format uh to help validate that like what we would like to do with modules long term will actually work so stream continues to pay dividends on what we’re trying to accomplish there a stream has a lot of potential but it is also a scary hydra but it also can like you can feed it a lot it has multiple heads

mike asks what are some areas of the sentence project that you think could use the most help or work well as we move forward i think that the sigs are going to be the most important community part of centos stream right because centaur stream is is the internal development for rel obviously community input to that’s going to be important to fix things and to fix problems and and for feedback but also uh special interest group content is going to be additive to that right so if there’s something that you want to get added to allow the community to use it then uh the sig is the perfect way to make that happen and then there’ll be a different there’s a whole obviously there’s a whole different process for getting things in the row we’re not talking about putting things into row but we are talking about getting things on centos mirrors and available to the public through centos stream in a sig process and if that stuff is good and if and if you guys want to work and get that stuff into real red hat will have a way to make that kind of thing those kind of things happen right they have they have partners and things to make make that sort of stuff happen but uh just getting it out in the community uh open sourcing your code getting your code worked on sigs are definitely where i see uh the the biggest impact in moving forward with centos i personally love to see more people start engaging directly with santos on gitlab i think for stream we have a really good system and a really good opportunity there where people can directly engage with the maintainers and send prs if they have specific things they would like to see changing in packages uh and i don’t think folks quite realize right now that that is actually a thing like i was mentioning before the hallway track that we had a really good experience recently getting a number of fixes merged in nine through the pipeline and i would love to see more people take advantage of that also because i think the more people use this the more the maintainers become familiar with the process and the easier it gets yeah i will definitely uh echo that i would love to see some folks engaged with stream as sort of like particularly screen9 for their workflows what do you need this os to do install it does it do that by default if not should it do that by default if it shouldn’t do that by default is there some tooling we can build to make it easier to make that change over i’d love to see people really look at what they’re running the system workloads are figure out how we can make stream adapt to those with as little effort as possible so that it can actually do what you need it to do in a clear and straightforward manner as those changes are going to stick around if they’re going to move into rel as uh devito was mentioning so we can get a clear defined make the system do what you want it to do the way you want it to do it process that then lands in the fully supported product so if you can’t run stream you can actually get these same advantages there and so i’m really seeing a lot of interesting workflow options available where you can influence how the os behaves but we have to get on record as to how we want it to behave

the next question is from robbie who asks some major hardware vendors have been less than enthusiastic regarding the support of centos stream does the centers project or the board have any insight on this is there anything the community can do to help i would say engaging with the vendors that you have a relationship with and both helping them understand what stream is and trying to help them get on board is definitely useful here uh in my experience there is still a lot of confusion sometimes around stream i definitely had to have the talk of like no santos is not gone this is still a thing and like no stream isn’t changing every day with more than one vendor so definitely that will help and i suspect going forward as these messages start sinking and there’s more and more vendors that start building against stream this will become easier yeah really the only thing that i see in stream that is any kind of problem for for anybody is is is kernels as as we don’t have the same kind of abi stuff that they have in release type kernels but the release type kernels are getting rolled into eight right now right so they’re optional installs but they’re not the default install because they have a lower version right but those things are available so you could uh because we need to test those as well right so so you could have some of the other kernels if you wanted them also uh if the issue happens to be modules or something like that there’s a sig for for kernel modules and the vendor could also start something that that that uh they use a kernel that they build themselves if that’s their sticking point so so and that’s the only thing that i see that’s in any way a problem in stream is is is the fact that some of the kernel abis move ahead uh during the during the period right that’s easily addressable if somebody wants to address it if somebody in the community wants to address it other than that it’s perfect i mean i’ve been using nothing but stream for my main workstation since we started the project and i’ve had zero problems with it uh for i don’t a year i mean yeah and on the user’s land front i’ve been running all sorts of various tests and the progressions against things in user space i’ve found i don’t know maybe a dozen additional apis that are in stream that are not in rel uh when you compare that to the roughly 1 million blinkable objects 12 is not that many and generally speaking they’re new exciting features that you have to specifically opt into for the userland stuff i really haven’t found anything that doesn’t run on stream or that runs on stream that doesn’t run on a rel uh they’re just they’re probably like they logically can exist but i haven’t found them so yeah basically if somebody converted your machine if somebody converted your linux machine over the stream you would be hard-pressed to tell the difference unless you did a bunch of research to figure it out i think i think that’s the the moral of pat’s story right there i don’t actually think you’d be that hard pressed but uh i mean the nvidia module uh pops to mind like that’s going to be a challenge that we know um that aside i guess when i read the question from robbie it was a great question but as the rel guy that does this day in a day out uh i read maybe more into it than it was intended because talking about a major hardware vendor supporting stream like what does support mean is one question the initial takeaway i had was if the major hardware vendor is intending for their hardware to be supported in rel then it will naturally have to go through stream to begin with so engaging our oem partners our isv partners that’s all activity that we’re doing on the rail side through internal mechanisms hopefully like pat and johnny have said things are working but the support thing is uh that raised some some eyebrows for me like what does that mean you know what does it look like and robby we can talk through chat or whatever and cover that if you need to yeah for what is worth one thing i’ve seen a few times is uh a lot of time people when they support them in qualification so they will want to use a product from a vendor and the vendor says this works on x and if it breaks on x you can send us a ticket and we’ll fix it one challenge i’ve seen there for stream is because stream doesn’t have versions it’s hard for the vendor to say this is qualified on stream because stream is a moving target to some extent on with my facebook hat on one thing we have tried and has worked uh is telling people okay given that we have composers qualify these compose and let’s use that as the thing you have qualified and we we had better luck with that approach and i think effectively composers are the versions for stream um of course this might not necessarily work for all cases but that is an option here and at some point in my san hat we have the same uh approach basically and uh you alex is around i think it’s a chat if you want to discuss about what was done around stream uh in terms of how we distribute it so let me know i will connect people together the next question that we have if there’s no more comments from the board um speak up now if you have any uh the next question oh go ahead i’m sorry i was just gonna say obviously the versions of packages in stream is different i’m not saying that it’s exactly the same i’m saying from a function i was just saying from a functionality standpoint if you weren’t actually looking at versions or looking for qualifications which vendors need in order to to certify things i was just saying from the from the standpoint of you running things and doing things on your machine that it functionally works just works right but it’s not going to be obviously not going to be the same versioning

the next question that we have is rel and center stream 8 has a ton of modules default to unsupported content right now php etc can we see default streams change to supported content in the future so people get the right stuff we have no plans to do that no it is i don’t want to trivialize the the question because it is actually a problem that we continue to iterate on and try to figure out how best to address on the rel side of things um but at the moment we have no plans to change the defaults so i just wanted to be clear is the point that is does it mean like the version of psp that we ship is unsupported upstream and absolutely must move on is that what this is about uh no i i understood the question to mean uh in some cases we ship a module with a default stream so if you type yum install whatever that thing is it will install that version and because of the way we have modules or application streams more in general because it modules a packaging type it’s not the concept that we’re really talking about those have a shorter life cycle than the full rail release and some of them are already out of support okay but you can still use alternate streams to get other versions that are still under support okay yeah that’s particularly complicated in database land where you have actual binary blobs on the disk that are your table layouts and you don’t want to get like surprised you’ve got a new version of postgres well that’s fantastic it’s faster it’s better supported it’ll do more things except the more things that it does are not read your database tables and uh that’s yeah that’s it’s complicated to get people the new bright chinese secured stuff and also not completely trash their production boxes that’s right and i will say this davide asked earlier about default modules in in real this is one of the lessons that we’ve we’ve learned and is something we’re trying to take into account for future versions so

we have another question in the q a if you have asked questions in the chat they have scrolled away and i’ve lost track of them so if you have questions that have not been answered yet please put them in the q a tab or they will get neglected the next question is do you think there will be a point where migrating from a centus linux 8 system to a census stream 8 system will start breaking that shouldn’t happen like it should just work i mean it’s hard to speak in absolutes i can’t really think of within eight i can’t really think of scenarios where this would be a problem now if you try to do an update in place from say centos linux to stream nine two years from now well that’s not supported but also yeah i would suspect that would not be as easy but for eight i would wager it will probably be okay i yeah again it’s hard to it’s hard to say for you know without looking at specific examples of of packages that things get moved to right you you can’t say for sure that that’s going to be the case but it should be the case because it should be fine because red hat’s not going to publish something that totally breaks moving from from rel 8.4 to rel nine or to rel 8.6 they’re they’re not going to put anything in there that breaks that right they have people moving all the time from one version to another and they have z streams that they support for periods of time uh and then people have to move off of that during the the life cycle so i doubt very of rel so i doubt very seriously that that that during the rail eight process there’s going to be something that that breaks an upgrade from rail 8.0 to rail 8.9 for example i don’t think that’s gonna break the same thing should be true for stream from from that standpoint yeah i mean i i think i agree with all of you i i’m perhaps a little bit more conservative and i would say if you’re planning to do that you know eight five goes out uh you’re on centos linux a5 and then a year goes by and you wanna update the stream please try do it on a vm perhaps first right get some data because if if that migration fails we want to know about it so we can fix it right um but don’t do it blindly yeah that goes for every migration by the way yeah for anything right i mean the world story is make sure you don’t always have good backups because you never know when your system is going through a hard drive so a good backup is always a wonderful thing to have

mike asks do you think users would be able to perform dnf system upgrade for stream nine to stream ten he’s not expecting this for eight to nine so i wrote this in the chat if someone’s interested in building this and helping make this a thing we would definitely be happy to see that this is one of those things that is technically possible because fedora does but someone needs to actually test it out five bucks for what doesn’t work 50 bugs and then do the upkeep to keep this working in the long term which is i suspect a non-trivial amount of work but if someone wants to do this or spin up a seeker on this that would definitely be appreciated yes and we we had the same question for seven to eight and we asked the same we said we need people to try to do it and we need help and unfortunately it didn’t happen so if you’re really interested it’s a good time to to to try to start to do something about it but we need people to show up and to to look at what can be done and it’s certainly non-trivial right for it’s hard enough to do for fedora and that’s like a six month window right of updated package versions and when you do it for for for centos or rel you’re talking about a five year minimum uh time frame right a three year to five year time frame so that’s like uh updating a fedora release to like a release that’s that’s uh six or seven versions higher than the one you’re on right now which is significantly different than doing a one to the next release packages don’t work data doesn’t work uh the configurations don’t continue to work across that vert that big of a version of uh changing packages there’s a like i would love to see a migration path will take you from screen mate to stream nine but as i mentioned earlier the the key there is for things that are written after disk what do you do with database devices how do you read the old data files that are written in the format that it recognizes how do you uh migrate your configurations for things like rabbitmq where it’s written in erlang now and then they moved it to an ii syntax and so we’re going to bring down your entire message bus because you’ve got a new package and that’s great and so getting those kinds of configuration changes where the actual syntax of the file has changed on you can be very difficult uh i look sort of quickly so if you look at the move from six to seven okay well we had the user move or the stuff and didn’t move to user if your partitions aren’t big enough you’re upgrading well which module stream do we pick to move you into for apache and so as we sort of look some point into the crystal ball for eight to nine well which module streams do you pick what and you’re going from uh as we mentioned the chat there you’re going from an unsub supported php module stream that has reached life well do we leave that on your box so that your web server still works because i feel like the expected result is well no give me the security patch the expect result is also well my app should still work and uh those are really opposed options but your your audio is a bit bad five seven your audio your audio is not great uh i don’t know if you can do something you are a bit too far from the mic i don’t know okay i can uh try and move to a different microphone i was getting the echo earlier and i switched to this headphone mic that’s kind of garbagey yeah you’re coming through a really chocolate it sounds like it’s network rather than microphone maybe a browser reload helps sometimes okay yeah it’s the ideas of micro are tricky but i’d love to see some work and basically a special interest group this is another example of where a special interest group would be great for for this kind of thing if somebody wanted to do that and wanted to have a special interest group that allowed uh migration of one versus the stream to the next we would certainly consider that uh uh you know on the board and see it by getting it approved and get people working on it but that’s something that is difficult and we certainly don’t have time to to do that as part of the normal uh main maintenance of centos stream in general right and i don’t know about you guys i used to do this for a living as an administrator and i would never actually upgrade any machine in place that had anything in production on it right it would always be on a new machine and i’d make sure all my data was working in the new place before i did a conversion um i i don’t know that there’s really a good case for doing major upgrades in place machines that are that are server based maybe work stations or machines that you use to run containers but not normal uh applications because of the uh the difficulty of moving data and configurations between different versions yeah i think from workstations this will be particularly valuable i agree for servers we have the same approach we knew can pave because it’s a lot it’s a lot easier but i think for people that want to run stream as they were stationed because it provides them an environment that’s more stable than fedora for example uh definitely the ability to have in place update to be really valuable because people don’t want to reimage the laptop every three years or so

we have a question from bala regarding container tools how long between a dev release to a center stream release for example podman 0 was released this week when will it when will it be available in center stream as far as where the real guy’s going to actually start looking at it right that’s not something that the centos true green team has anything to do with yeah and i think having an answer you know at a higher level might be helpful i mean this particular package no i don’t think we can answer that but but understanding how the rel development process works i can speak to that that one’s actually pretty easy so when a particular version of a package or update will land in stream is entirely dependent on the rail development process so container tools has their own road map that they do for rail customers i actually don’t know if it’s public or not i can try to find that out but essentially the plans there are they kind of rev it much faster than the rest of rel and they have multiple streams at least in stream nine stream mate they’re kind of revisiting how they’re how they’re doing this an upstream release into when it gets into rel sometimes it will be never sometimes it will be in the next live stream and then we would see it land in centos stream first and sometimes we skip upstream releases right depending on what the product roadmap looks

like marcin asks will sigs continue to produce packages for sento stream 8 after the centos stream 9 release overlap of content production for main repos is obvious but for sigs not so much i can take this one so it obviously depends on the sig uh i can say that for hyperscale uh we are currently building for 8s and we start the building for 9s and we’ll continue doing 9 when when we can actually release packages uh i would expect we will keep everything up for eight until the eol for eight for stream eight i mean my hunch is that you will probably continue seeing a decent flow of packages in for eight for the next couple of years uh after that i would expect it will probably taper out because most of the developers and most of the work will have moved on to nine but if someone wants to continue doing bills for eight there’s nothing stopping us for it and if somebody in the community is particularly interested in a specific package um i think we could definitely try to keep that actively built for eight um going forward um but then everything is gonna have different policies on this i think another possible answer it’s the same answer but stated differently if this is something that you care about then come make it happen you know step up and be part of the solution and that all of our sigs are very welcoming to people that that want to come and and help out and so if if the sig does not have the resources to continue a stream eight build and you want that to happen then make it happen in doubt always create a bugzilla and uh worst case you will be told no but at least you tried yeah and i think honestly for at least from what i’ve seen the the bar for like oh we need this package built we didn’t say hyperscale because like i don’t know we need this version of crap updated because as a fix it is not a ton of work uh so like if someone asks nicely we will probably do it if someone wants to contribute the work that is even better of course

we have time for i don’t know one or two more questions if there are any i’ll take this moment to remind everyone that board of directors meetings are now open to anyone that’s on the census develop mailing list and wishes to join them the when the announcement goes out for the board of directors meeting there will be information in there about how you can request an invite to the meeting we don’t just post the link to the meeting live online because that leads to spammers showing up and doing unsavory things but uh it is open to anyone who requests that the board of directors meeting will be held on october the 13th at 200 utc i just sent the email to centos devil i was like by one day sorry but it should be it should be in so we are still building the agenda for the next one but feel free to bring subjects if you want

i think there’s a few more questions things yeah i just saw one come in um and this is this is a good one what is the status of rel build roots for cigs and there is a ticket about this number four hundred and that might actually be a question brian stinson might know something else yeah that seems like an infrastructure sig uh question yeah to be honest i don’t have feedback on that i can’t reply yeah i saw them on the ticket that we were fine with doing it but i don’t know where things are at yeah for the recording brian said in chat that they’re working on it and then they have to change some of the ways that we consume content in the infrastructure space

whoever anonymous is they have very good questions true story those are some really great i think we didn’t reply to martha in question did we the second one will sig continue to produce package for c8 we talked about that okay sorry i thought it was another kind of question okay all right well we are almost out of time so if you have a question here’s your your last chance