Flush to Data

We mix Water, Poo, and Data.The Flush to Data podcast invites conversations about the data, models, and science used in wastewater engineering. Your hosts are Jörg Rieckermann (Eawag, Dübendorf, Switzerland) and Kris Villez (Oak Ridge National Laboratory, TN, USA). The contributions of Kris Villez to this podcast are a reflection of personal opinion only and are not related to any project, study, or opinion at the Oak Ridge National Laboratory or the U.S. Department of Energy. ---- Credits: ---- In our Trailer and intro we use the beautiful music from the "Obliterator" Amiga game by the incredible David Whittaker https://en.wikipedia.org/wiki/David_Whittaker_(video_game_composer) we also use creative common artwork distributed under a CC BY-ND 2.0 license. Check out our homepage for more info --- https://flush2data.gitlab.io

All Episodes

Flush to Data

Episode 04 - Open Science

November 23, 2020 • Kris Villez and Jörg Rieckermann • Season 1 • Episode 4

0:00 | 1:31:36

This is the fourth episode of the Flush to Data podcast. We start of discussing open science. We venture into many related aspects such as scientific reproducibility, scientific career evaluation, software versioning, and the distinction between the scientific process and academic institutes.

Hosts: Jörg Rieckermann and Kris Villez
Guest: Juan Pablo Carbajal (Hochschule für Technik Rapperswil, Rapperswil, Switzerland)

Links:
* Juan Pablo's web page: https://sites.google.com/site/juanpicarbajal/
* OSF - Open Science Framework: https://www.cos.io/our-products/osf
* A publication on the scientific mission: https://dx.doi.org/10.4161%2Fcib.1.1.6285
* Open peer review: https://peercommunityin.org/
* Book "On Fact and Fraud: Cautionary Tales from the Front Lines of Science ": https://www.jstor.org/stable/j.ctt7s7j4
* Goodhart's law: https://en.wikipedia.org/wiki/Goodhart's_law

Episode guide:
[00:00:19] Typical approach to sharing of scientific results today vs. new approaches
[00:07:20] Is the product of science a pdf document?
[00:11:15] Juan Pablo's approach to producing and sharing results
[00:19:50] Pre-register your study and methods to mitigate the risk of (inadvertent) p-hacking
[00:27:10] Utility of open data, FAIR principles, copyright, licenses, sensitive data
[00:45:50] Open peer review
[00:54:10] Evaluating scientific careers
[01:04:55] Practical measures to achieve the goals of open science
[01:07:40] Science is ...
[01:08:35] Closing main session
[01:09:10] Start bonus session
[01:09:15] Is open science a sad story?
[01:11:50] Publish or perish
[01:15:25] Creating knowledge is easier than ever - Can open science be weaponized?
[01:24:04] Why do we publish? Why do we stay in the system?

Jörg:   0:01
Thanks for listening to the next episode of the "Flush to Data" podcast. My name is Jörg Rieckermann with me today are my co-host, Kris Villez and our current guest, Juan Pablo Carbajal.

Juan Pablo:   0:18
Hey, Jörg !

Jörg:   0:19
today we would like to discuss open science, which is a topic which has gained a lot of attention in recent months and which is known in many fields such as physics, but has not arrived fully in the wastewater field. So we would like to um yes. Tell you are recent experiences with it. Andi, like, maybe also motivate you to follow us down this path. Like as a short introduction. Like we are still experiencing the Corona crisis where epidemiologist share data and models, um, to make predictions, what is going to happen. And for this they like, um have, as far as I know, moved from, like the standard model off having peer reviewed publications then like screening literature for the important information after the fact to really aggressively publishing or like making the new knowledge. They have available either new tests which they developed in the lab, on home pages or heavily working with pre prints, like a good or bad quality. Whatever. Andi, this is like something which we would like to discuss today. Eso Chris, if you could probably tell us as a kind of experience your personal thing, take us on a journey like when you published your first paper. Like, can you tell us about the process like,

Kris:   2:01
Well, maybe maybe we should go up it, uh, Mitt back further to my first experiences with open software. So that's good. Yes. And my PhD, we're tired. We're talking 565 4004 through. I wanted to use this technique, which was originally published in 92 I think. And this was, of course, a time before the Internet took a stronghold on science so so I could find the paper. But there was no code. There was no chance of finding that coat. And the researchers themselves didn't have the code maintained eater I figured out later, so basically implemented terra matted from scratch again. And this this was a very difficult experience. And for me that highlighted that this was not an experience I wanted to impose on the reader's off my article that I was writing on DFI Chur articles that I was going to write. So for me, that was probably the most important experience, at least initial one that drove me. Look, when I write code, I want it available. And I wanted to run hopefully with minimal efforts on somebody else's machine. No, I was basically self learned. And the suspect? I don't necessarily achieve the goal. Always, Um, but it was the first of them in this direction. So So what? Almost every article I think US published in Stan the supplementary information would have some package that would at least allow to reproduce the results. At least that's always been my intense instead. So But that's not the same as open research per se. Um, and I've learned a lot from going Pablo on how to actually might make a coat findable or searchable for a while. I just added it to my personal webpage. That doesn't necessarily mean that people can find it. Um, they would need to know we're to look essentially. So I've been using good lab a lot now for in recent two years. I was thanks to home power s so that's one aspect. Open software has been part of my life for a while. Open publications really new to me still. Ah. So we did do in a tent. Ah, a couple of attempts at the air work the experience I had. So I had put a pre print on ah, archive, I think. And then I Once it was accepted for peer review publication in a journal, it went to the total editorial office. And then I got a whole slough off emails about plagiarism and how I was basically copying existing work.

Jörg:   5:13
Oh, your own way

Kris:   5:14
found on the Internet. And of course, they found my own work. So then So that's of course, that's something that can be overcome because, no, I didn't violate my own writer the order, nor did I violate any of their policies. Um, but it was a bit of a hassle. A bunch of emails, I really would would have Yes, there was not to have to deal with. So I think when we talk about open science, there's obviously some kinks they need to be ironed out. If really want everyone to accept this as a practice,

Juan Pablo:   5:53
you touch their toe to important anyone is reparable civility at any level of the scientific process. You call it so far Concordant experiments A Your mental Aziz. The theory also and how you get to this equation, how you get to the small M on the other thing is the anything about open, which, in my personal experience, myself and the people in tack it was at the level of the scientists again. He was never need because you could always right a person say, Can you send me your article and that you will never They will never say no on. Then I never In my experience, I never got somebody say No, I don't have paper something that then they This was kind of institutionalised in blood from liking a research gate and things like that. And then, of course, there was. The archive stated that sucking, appearing. But I think this thing about the Open publications is in. He has his concerns, mom's institutions and how scientists are assessed than the actual That's a by scientific values off course. I I do agree that it is good to have opposite, I think Sci WAAS already open, not through formal means. That was probably so means the we're not used to assess people on them. But the only aspect that approximately for me it's a core value. A scientific body that's nothing to institution.

Jörg:   7:22
Yes, but like, I think we're touching a lot of stuff he already like. We talk about the like, ethical principles. Like we talk about technical tools like it lap, um, version ING systems write code repositories. And I would like to, like, pick up the readers or like the listener sorry from the West border community. Um, big bit by bit, you know, like, Ah, and I think my own experience like matches Chris and maybe a calm Pablo's like that. When we started in science in our field, it was about the Let's say, the pdf right Like you. You you made a nice study. You you you had insight and you write your conclusions in the pdf and that's it. And then year by year or time by time, like we realized that the pdf is not the knowledge which we should like like propagate as a scientist. So, um, but when I when I look at, like what in off wastewater field, what's happening like I still see that it is a lot about the pdf. Now, um, maybe one. Probably. You can tell us like how this works in different communities, right? If you like, maybe even from the first contraction idea or the first draft. You know, like, tell us about your experience with pre prints. Maybe with version ing systems. How this works with, Like, what? Chris mentioned the plagiarism, right? What a pre print is and what the post print is. Um, maybe you can have ah can give us a good example.

Juan Pablo:   9:03
So I don't I can't say generalities of something's a pilot. It's on. Fields are very similar, you know, these are basically there. The period is definitely a very valuable document, but it is never seen us an institutional weapons rights never weaponize so in. It's very common that you will distribute it even after you, even before you know where you are. Probably the only if you're gonna probably you justice with your work. When you're working problems stop that. That's very common in those communities and data the same and stuff like So this really is a community were really naturally on what now the official science inspiring toe get, and then you have all completely from something reward things Where the pdf, the document This article your rights like a basting card. My, you really ask Like the medication on the value these documents have already know. They're just kind off like a a infomercial off a what you're doing in in Europe, basically on them It is there, of course, many different places you ever like in many from ways. But in my experience in road was things that basically articles are a not It's not like a core thing. They do it because they're evaluated on that. But the quality is low on, the amount is huge, and they're basically it's Morris. How well you're not because of the things you have invented. Your God, Yes, they're this small, relevant, that thing so far and gadget. So it has a lot more weight. Mm. So in Yeah, it is a lot with the culture off the community, this anything on what is a meaningful stuff. But some of these photos, the berm ice, is that they become really wasteful for the way the system, he said. Because of the end of the day, a provocation means money. And most of them that means public money on am Yeah, he If it is there not too many. It's not a problem. But as soon as you start accumulating volume us, we're doing, you know, you better start worrying. Okay? How is this money spent?

Jörg:   11:17
Yes. No, I think what I wanted to do, like toe hint at is that in the community where you've been moving before, there are a lot of valuable things, right? Like the versioning thing, Or like putting the first draft of a paper on a preprint server. Maybe I want to establish a priority or to have other people that you know what it's working. So maybe from the last paper, you've been working on, maybe very practically. Like what? What did you do to get this published?

Juan Pablo:   11:47
And so my president Lee really like what you say As soon as I have a document that I consider readable. It goes in some sort of mark I in the archive.

Jörg:   11:56
The arXive is what?

Juan Pablo:   11:58
they're officially our online repositories, which have an index and are searchable where you can a a ablution meta data about your pdf on the pdf or the sources to build the tedious, Um, And nowadays you also get like, a unique identifier, so you can do a Perma link and stuff like that. But it was essentially it wasn't like that. Actually, when I started with these kind of these, you know, you know, so, baby, yes, where the thing at that time And you know what the baby s is? But there is before the Oakland that basically are like telephone networks, where you come a make files available and you can get it from there. So that was the way I was getting. For example, papers when I was doing studying physics and was telephone that you would call a number. And then you connect your computer and you will be able to download files from there. I m so. But, uh, yeah, So my process is basically that that's one. There is my papers. Always online. The sergeants are always a man doing the whole process. I like it. I always hope that somebody you know, if I'm making something that's interesting now somebody will come in and start contributing from the carry on for me. These has to lose all 11 process off. More people can, you know, tell you something. The battery will go and he never happened for me at every stage. But I had some very good excellence. When I put these PdF, let's say this readable version of the document online and I have been balding Inserra really fruitful discussions on they being able to make contacts and everything. Based on that again, this question don't go completely. They go completely away from the publication itself. It's more like content oriented un and what is good. What is wrong. There will come. In truth, it's got opinions. So

Jörg:   13:53
basically, you develop your idea online. Like if you get input from the community, fine, you acknowledge it. And if not, you proceed, like with finalizing the publication. And then when you think it's good, then you submitted to a peer reviewed journal, Is that correct?

Juan Pablo:   14:08
Yeah, so now these become a lot easier. Before there were some may a peer review journals that will not allow you to do that, what I was doing, and although there was no legal basis to do it, they will just don't like it basically helps of you. One of those. If you have they my a workflow. They will not accept it if a publication. And But this has changed lately, because, I mean, it was full of hold this thing. Yeah, they there were so many journals, tried to apply some sort of like a retroactive copy right on your money. And that's not possible. Am as far as I know, there is no m a country that has laws that allowed to do that. So So, Mike, my other beginning, my selections were very much constrained. Basically, toe his owners of who lacks of this flow. And, yeah, they were usually conferences. They will, of course, have no problem with that. I m on a on some. Yeah, in physics again is very common. Even there some genital use those repositories for the review process.

Jörg:   15:20
Yes. And so once once the once the paper has gone through peer review, you incorporate the comments of the reviewers and then this gets put to the journal layout. And that is then, like the post print, right? Yeah. And then what happens? Like And I'm also thinking about like, um, the data which are connected with the journal and maybe the cold.

Juan Pablo:   15:47
Okay, so in when? So this meta later, this extra materially supplementary material was tried to get him to wear somewhat in the repository where I developed the code aim for the article. So let's say the source files for the article and there will be already links to the other repositories that have the material I'm using. So this will be connected from the very beginning on. And then I will do some that the repository or more comprehensive repositories nowadays, like oh, accepts that allows you to include everything preference data, A all your recommendation. You could put everything together, and you can create that USAA a digital object and share it on. You can, of course, link it on your peer review publication. Do you have to put up, you know, before a at the median? Usually, yes, when they're doing the editorial corrections, but, um, yeah, that's what it is. That's a rather mother, and I would say from the last four years, kind off by a workflow. It was not like that before. Like that before I was trying to keep it myself link and on

Jörg:   16:56
the way back in the way that Chris mentioned like on personal home pages. Put a sees V file with, like, data. So

Juan Pablo:   17:02
yeah, exactly. Yes. A personal when I could I was always trying to have loaded the promise that is still not common, that they sometimes journals will not provide a link to this planetary material. Although you have uploaded supplementary material s so it's unclear how you people will get that supplying their material that waas mate emcee the problem on them. But again, I think everybody was living this And now we have really good solutions. I particularly like the OSF solution. Mending it really much is the workflow that I have a allows for collaboration again. I think the other the people may know is a Sanogo. Cinema is like one. Is it similar? Yeah, Another right Is it that the repository is a one shot thing? You can create your a that object and then you put your mind. Andi, that's it. You and him and it's ah one men work like if you're cooperating with all the people, one person needs to do these by them. So it's not like collaborative development on the tell object in this goes completely against away. I usually work on and you know, different people who country different aspect and centralized in one person was. I mean, I don't like centralisation in general. I am so a yeah. This world for personal for me is very a abnormal while a something more like also follows to do collaborative development of the least a locket. And then you make a snapshot off the general you can publish. So when you make your deal why, it will be a snapshot off the current state of the objects or for a future, a publishing if you keep development, can keep developing on the same this doll object on them on the new snapshot will just be the updates on the old assumption will never change.

Jörg:   18:53
Yes, so we're told the always f just for listeners that is the open science Foundation.

Juan Pablo:   18:58
Well, Frank or 80 school opens, I will frame organ. Yes, always stepped up or

Jörg:   19:04
Yes. So this is one thing you can personally recommend and then say Noto is like data repository right, which is connected to discern. And there the the argument is that certain produces, like, I don't know, 700 megabytes per second. So the couple of megabyte, which researchers add toe their data doesn't really doesn't really matter, right? Yes, just a

Kris:   19:34
quick correction. The always stuff foundation is that osf dot io

Juan Pablo:   19:40
i o uh Yeah. Okay, good. I

Kris:   19:42
don't orkest something different

Juan Pablo:   19:44
comes very

Jörg:   19:46
Yes, I I also familiar with the OSF. I connected it mostly with, like, some challenges or right rewards they offered for people who pre register their study. And this is like which should contribute to the quality of your work, like maybe less in our field, maybe more with surveys. And so where you Yeah, you must make your hypothesis public basically and then like no, like get tempted to tweak the results of the study to your prior basically.

Juan Pablo:   20:17
But now that you mention it's funny. I've been collaborating with these people in in the the sanitation technologies department on they are kind off going a head a front into, you know, like fitting data with moles on and on in the very There is also this problem that if you just call it, they're gonna wanna fit my data with some model. And the space is infinite, right? And you run into the same problems that psychologists will run when they do not resist early. The mythologies basically end up doing some kind of be hacking or something. So and he's positively general to have these or illustration whenever you want a feat. Moles to data or you some data analyses that as the theory that makes sense.

Jörg:   21:05
That's very interesting, Chris. Like, uh, give us your take about that. Have you ever pre registered to study?

Kris:   21:13
No, I have not. I know it's very typical Onda medicine. You procedure? Yes, it's proposed or a new medicine. You have to say what it's supposed to treat so that you cannot just randomly distribute it and then posted to say all it. It seems to help with obesity. Yes, but in the meantime, you tested it for 20 different diseases so randomly you you would say at least one expect with a 5% P valleys of speaker. Yes, at least one should pop up randomly, so But no, I have no

Juan Pablo:   21:51
you mentioned less than a anything about experimentally science. Right on experimentally sign. If you understand correctly, it should be based on prior knowledge you have upon you before you. Your experiment, right? And I think saying Okay, how you gonna do with your experiment in space and the pregnancies? You need to fix it at that point, of course, and you can look it by its informal looping the things to learn from the new later in like a posterior. A experimentally sign. But it is very important. It's also efficient, you know, because you can focus. Okay, I say when I do this thing, let's tow the staff on then allegedly more efficient not to die birds in all the possibilities, right? What you can do. And as in these, especially in the students students, you will start moving some data. They never they go, you know, breath First on day never reach further because like so many options and so many models and so many different names for me from models and aim is why so? Basically I've always tried to kill us, discuss for the understand of the problem and let them pick a little bit on whatever they know At that point, What will be the approach? Onda? Stick to that and see if he fails. We'll learn something. If you have some favorable, so learn something different things, but yeah, they're no longer doing that.

Kris:   23:11
I do remember one study as a reviewer where I thought it would have been really nice if if this was common to do. If if these people have set out or the journal would have allowed to say, Look, this is the hypothesis I'm gonna test and then execute the study and the situation waas where they they basically wanted to compare a novel approach to an existing one based on assimilation and assimilation. Self was proposed by somebody else. But then it looked to me that they modified the simulation, notified assimilation such that this new fancy tool was gonna do better. And if, as a reviewer I had, I would have had the chance to to inspect that I would have said issued at least test those methods on the original simulation Also So So, yeah, I think nothing. If if we could come to a procedure like that also in the ways whether research especially where models are involved, nothing that would be that would be great.

Jörg:   24:30
Yes, but like still the typical study looks like Oh, we have, like, two treatment options. You know, like we make these experiments and this one is better. Or we have this machine which analyzes our data. Like, what is it clustering or like, Prediction. And this machine predicts better. You know, it's like, far away from hypothesis, like politik ation, right? And I promise,

Kris:   24:56
when there are four methods and they only report on two and they forget to report that they tested two other ones.

Jörg:   25:05
Yeah, but we're engineers, right? We're trained to provide solutions. Were not, like trained to falsify hypothesis. I would say that's what but I would love to see the first wastewater study pre registered on the OSF

Juan Pablo:   25:22
again. I don't know if you need to be in a former preregistration the way I do it, For example, I work many times, you know, I help people with the data. And then what I do is after I have discussed with them and somehow, you know, exchange and I come to an idea when I you know, in my head somehow sort off mother off. The situation is building up. I will write that the document and send them this document to know what I'm gonna do. One that kind off a compromises me. I commit me into that line because it's what they expected. And it's away Everest. And for me the was helping because, you know, every time, say, Oh, I could try now the ex other metal because I see what is going. But, you know, I said gonna stick toe the only thing I can push it to the end. And I think that for me, it's really helpful. And so even beyond going in the formal aspect off it that they make sure that we do better science. But on the productive side on an example, you bringing things quite commonly. You have, like, three medals to do something on. Then you want all of which one is the best. The productive way of going is if you're gonna decide on how you're going to compare them before you look at the data off these guys, that's based on what you understand off this mental A. How do you think is a proper way of comparing them on? Go with that on based on the results reported, the results of that care for and then you say, Well, you know, in the Brussels will realize it would have been better to like this when you have your second resulted, you have your undated benchmark. But then usually this prosecutes all a compressed let's say, on only the final A benchmark. Actually, many duration is showing his tends to could be revised.

Jörg:   27:03
Yes, I fully I fully agree, and I would like to deep in the discussion. But if I look at the time, I would move a little bit to the open data aspect because I think that is like with the tools like C Note or something that is pretty new. But we also had, like, deep discussions on limitations and bottlenecks, especially in our field. Andi, maybe home. Paolo. You can comment like a little bit on the fare data like the fair principle, which means that data should be findable accessible, Inter operational, yes, and especially reusable. And I think what is also a bit less considered in our field, our licenses

Juan Pablo:   27:46
no so licenses incorporates an issue because it's not an international think right sense of the country and the loss of the country, how these things gonna be interpreted, and to the point that in ST Salon is very special thing, because that is a data is not properly defined, you know, like a thing. A. There is no legal definition for it. So it's really a open there what it means and sunny because student and you will find people saying like, No, you have to do this and and you have to do this you know reverb there highlighted. You have to do like this when you study in India. Actually, you don't have to undergo. There's nothing, There's no legal is re a personal decision. And some people, for example, bulls force on data a like community or secede. Oh, basic public domain Republic domain and I understand why they do it. And I every Emma line without things. But I think it defeats the other purpose, which is that you somehow want to get some sign academic credit for this effort. You put on the data on making it 60 destroyed that because you rely again on the goodwill on the understanding off. Other people to cite your letter said, if they use it and while other licenses like the CC not civilized and city bias area and things like that, yeah, somehow will inform if well presented the use of the data, how they should treat the acknowledgement off that data. They're all completely permissive. A licence is that you can do almost anything without a meme on. At least somehow warrant is that they you know, it provides a path to inform the use of the data off what they could do on how they should decide you. And that's why I use that. But again, if you go in many institutional in many solutions, that will say you have to do with CC zero and again I have, like they will say you have to do it. You don't have to be like this is where they decided there is not a legal around.

Jörg:   29:56
There's yes, But I guess there we can we can get like into different categories like, for example, a typical wastewater lab. You know, you get wastewater, you put your own pH probe in there, you generate the data yourself, right, And then you build your own model, your own software, and that's all clean. But they also many studies which, like with with case studies, right Andi like, for example, if we take the sewer system of Zurich, which we used in some paper, and we would like to make this study reproducible. There's the data. Don't belong to me, right? I have, like, a kind of use license, more or less formal or informal from their own off the data. But, um, what would be a good way to ensure reproduce ability in such a case?

Juan Pablo:   30:46
I mean, there many levels of progressivity you mentioned already these fry and so I for a depressed. I cannot imagine a situation in which a the own data will be actually needed in a scientific context, right? Maybe an engineer context. It is more relevant by significance when you're developing a mythology or a when you want to pass some method, the actual data years and it's not that crucial. Right? So if you could get let's say the killer data, you only was able to get the court. Okay, This happens in the last paper published on Wind Energy. The killer data set is private. It's from a company for energy companies so that we couldn't share.

Jörg:   31:31
Can you write that? I did not copy that. Can you repeat that, please?

Juan Pablo:   31:35
So we published a few months ago a paper in frontiers about a or optimizing em the use off a wind resources toe, reduce a or to increase energy availability. Tried to compensate basically for the patterns in the demand. Using were distributed wind sources on them and a We basically need to observe the demand off this energy. So the imam we got from a company on they say you cannot share this data. This is say so he was We couldn't share it. But our methodology does not only apply so that a demand data right there. Waas really like Okay, it's good because we can prove it. In reality, we do later. But we also provided a win synthetic deficits that we created that are basically, for example, we picked up from the demand some partners and then we created data with similar parts and stuff like that. So again, at least the way I usually work, I don't see any situation in which Yes, I mean, this data set is absolutely the key thing. M. Andi later said, I could imagine are like that are usually public. Tomei like I don't know some of the sequencing or stuff like that a or that usually will be available if general drugs, Maybe that formulas, that will be a Yeah. I mean, they're so again on the retail, but in general would say, it's hard. I cannot imagine a situation in the scientific framework with you. Actually, that data said to make your point and different, you can have used that data set to make a great paper. But they refuse is gonna use that. That I said I used some other that paper still good, right?

Jörg:   33:25
Yes. No, it's probably more about right. Maybe infrastructure, because we work a lot with infrastructure, you know, like, imagine, like a mechanistic model of atomic power plant, you know? Yeah. Maybe you get information from pressures or like certain evolve settings, which should not necessarily be available to everybody.

Juan Pablo:   33:46
I mean, there's a thing about an only mice in the data that is very it's a lot off our work. We have to do it in one product now, and we're getting some burner information from Boyz Burners and then we toe blindly inventor process toe. I don't my sides data. So basically the client, it was like, I want to give you the Zeta, but you know, he's sensitive So you need to have money. My sister will have to write scripts and send them so they can run them on. Then later. Then send us the results. And so I in It's a lot of work. And sometimes, yet you Sometimes people are just afraid, right? They say sensitivity, just in case. But they actually don't know either. The sensitive. I mean, I believe in in the future. Nobody complained. The future in the future. Oh, possibility that can become sensitive.

Jörg:   34:38
Yes, yes. We've been discussing a lot about sharing data from a sewer system, you know, because terrorists can go in if they know what diameter this ways, they can rob the bank. You know, if that they get that information from, ah, more model. But like, um, recent experience with practices like practitioners, um, they won't see the benefit, right. Like there was one case where the city of Vienna, they got a price like innovation price because they made their sewer system available on the Internet. I can't download the whole thing, but like for a city district or a couple of roads, you can get the information where this was, how deep they are and I think the biggest benefit is the is the crane driver. You know, like the construction sites, like the guy in the road can like. If they have to take up something, you know, they can look on their mobile phone. Oh, how deep is the sewer? How? When do I have to be careful with my machine? So they save huge amounts, apparently in not occurring damages. You know, before the guys were blind that we're digging and then bomb, you have to repair a sore for a couple of 100,000. So in that sense, open data can create a lot of value. Yes, and like, I think like getting synthetic data is is a very interesting concept. And maybe another interesting concept is that replacing the mechanistic model with a surrogate model. So you do not share the model off the atomic power plant, but like some machine and you're a network also, which gives exactly the same response to the same input. Yeah, right. So So you mask basically the processes. And yet,

Kris:   36:20
since the community away for the committee has course already delivered, that is the so called benchmark simulation model which exists and It's meant as a platform to test monitoring and control strategies. To a certain degree, people have been convinced of this and I followed up

Juan Pablo:   36:39
to them. So I usually feel this preoccupation is a not a a grand grand because the drive off the potential wrongdoer my will not be It is No, we know these That is not a hinder by already security measures we have, right, like, really measures a government that you can have access to data on the wrong door stable toward to those things. So I don't know if it's really on the side of the scientists. A you know, be of this world that will be super easy to break em on the other side. Hey, what I always hear off the people that seriously care about the data is this very simple. A statement which is insurer on skins. The recent way they considered the data sensitive is because they know if they when they say okay, for example, that's have pump breaks right and you have some insurance for the pump. A. Of course insurance says something like under normal usage discovered on the normal usage. So if they're able to ruin your data and see that, uh, you see, your so many times when about specifications are like that, they can say we will not pay you the whole thing, for example. Yes. And I am. So this works in motion. That's a really fear a about not releasing this information. They only sensor data on them, in which case has not been about anonymous. Nate, About getting rid off this pervert perversity of a relation between insurance and yes and insure

Jörg:   38:17
must liability stuff. Yes, exactly. This is

Kris:   38:21
in this context concept. Off differential privacy could be useful. Of course,

Juan Pablo:   38:28
I don't know the concept differentials

Kris:   38:31
differential privacy. So her policies will be used in the upcoming census in the US on the ideas of following. So something in this case is to survey. Of course, Surveyor asks, but your age and they give up before you give the answer, um, to services. Ah, flip a coin. And if it's heads, you give me the true number. And if it's Dale, you give me a random number. Andi. But you never don't. Your surveyor if the coin was head to tail. So on average, from your population, 50% of the time, you will get the correct number and 50% of the time you will get a random number. And I just just statistician, because you're only interested in the population, you can store data because the background distribution looking sort of subtract er in the background is the older details right? Or all the coins or the head. And as an individual, if somebody draws you to court, consist Hey, you said you are 23 years old. You can say no, No, um, the coin was stale. I didn't lie. I just

Jörg:   39:58
gave you the random number. No. So that's that's coming. That's why that's why they they don't really they they apply that in the wastewater sector already. Like from that From that, I didn't know that, Chris. It seems that the U. S. Is way, way ahead of the world again. I do

Kris:   40:21
remember the conversation from ah stands up with a sense of company CEO. I went to China where another company had sort of copied their sense of model, but and that since it was a random number generator, good. But the client really like in this case, was that the measurement was never above the permit limit

Jörg:   40:47
on they really valued. Yes, okay. But like summarizing the open data and the fair stuff, you know, find ability. Let's see Noto accessibility also, if we have Internet, um, otherwise, write it a letter to the editor or like the author of your paper, and asked him to send you paper printouts, a region visibility. We have copyright. We have licenses correct on regarding interoperability. Is there anything to say?

Juan Pablo:   41:26
I mean, they re 70 some only the permission You give story to re use it. But also how useful is that itself? If you are blowed, you know, like you're a n Columns CSP and say That's Molleda that cannot be reuse. Somebody will license and what cooperate you put on inciting Reusability is also all day made a daytime manuals and examinations that you provide with your later, and so the structure of your off the detail of the That's very important, and I think that it's what makes your data management plant in a way. That's what makes a anything expensive in money and in time, and why people who do this properly should be for me. These people are really useful because getting a useful that I said It's something that sexual a lot of time And so it's definitely time for me that we start valuing these people in these contributions. And I think their operability many a times ask about it is made an isolated too soft for, you know, probability and platform. That means if you have something in paper, is nothing there parable because you can only do it to a human or something like that. And if it digitally will, depending on the four months you're using, I am. But, um, again, you're probably so for you still cities that sharing. Um, let's say people say Is everything open? But then toe open these things you actually need Probably software. You okay? How open it is, right? Yes, but

Jörg:   43:01
this is very common, especially in the engineering field. You know, you have your windows software, and then that's it. But do you have any any good research like resource a book or like something which which points to reusability something which our listeners could look at if they're interested

Juan Pablo:   43:18
recently in terms off a support for the user? The potential users of the data there is already nice a thing of the class. Income from the data actually come from the harbor. People with this the Oshawa, the open heart worry open source. Her were a community. They have developed best practices. And this purse parties, you can easily translate into data as well.

Jörg:   43:39
Okay, so you would provide the link and we will put it to the to the page. Yeah, very good. Then

Kris:   43:47
one point I want to add on the reusability. So if if it's 30 years ability for the same question on the same purpose, let's say we have everything together. You can do that now, but if 11 thing that concerns me sometimes that data is collected for a specific purpose. But in the future, maybe in five years of 10 years, somebody wants to use that data to answer a different question. Andi, increasing the chances are making sure that people know what the original purpose was off that data so that they taken reflect on weather. Their new question that we don't know yet can actually be answered with that data that I think that's a tough one.

Jörg:   44:33
Yes, it is a tough one.

Juan Pablo:   44:35
I e I think if you have a good description of how the data was acquired, this is definitely a part. You could be right And why you were according to say, the word works your experimental. Now see what you wanted to tow, approve of this or whatever has different. Very important. Yes, it's very important, especially because when one is taking vigorous a lot off some, like in everything, right, this is the final object that to get to that there was, like a 1,000,000 little decisions of war meat on I'm. Hopefully one could document all those sickle decisions in practice very hard. A. But in many situations in those cities is where they realize, right, like, yes, the state that will be useful for you are not a an m.

Jörg:   45:19
Yes, but like, for example, flow meter right. You get 50 minute data. You don't know Where's the time stem? Is it like the beginning of the interval? What's the value? Is it like an average over the 15 minutes? So is it like instantaneous? Value are It's exactly super. It's super difficult, and that might be might yes, like if you get average data, but your model critics instantaneous data. You know, it's really important to know. Yes. So that's like a very good point. Chris. Um, one thing which I have on my list, which I find very important, is the open Peary. You and I would not spend too much time on that, but I think this has also changed a lot with technology, right where you can see the reviews off your paper. In some journals, you know what it's like, not like hidden anymore. And it's like a kind of essential part off the publication process. Um, maybe Humpy, can you can you share your experience with the open peer review?

Juan Pablo:   46:23
So I'm actually haven't would be important only open review for me. It's very a few, only a few papers on their journals on the Once, and actually I know them. When I waas, I enter a rod because in other committees, not that common a open with you And but again, you feel I always will be from a point of view off learning, personal learning. My many times I have understood better the paper by reading how was commended by other people that know more than me a city so this conversation when its put like that between the reviewers and authors. Yeah, I can contain a little thing for me and off course. You probably have a lot of the things that are, you know, setting my publication or things like that. But when and you have these articles in which the actual content is discussed and again it's like looking at the planet off experts right years, you learn a lot by their Not so they we have not touching on scientific misconduct, which many of these things were discussing. Also aimed at preventing those and one of them is fraud on open science policy in general, basically releases to almost cedo the possibility of fraud and which is more common that people would like toa except this ah, book a written by a person who retired at the University of California School on Fact and Frau. I'm from on and is basically a collection off. This person was director of the university, and, you know, he collected all the cases a keep more documents, many situations or fraud on the batter's common that they are, and they it is incredible to see how often at this time and again, is this and perversion a uto a natural miss inclination off the scientists already something the least even by by the rules on system we are able and I don't know Yes, we'll have the end. We have many off. These likely the mechanical torque and the race and the counting horses are many historical examples of fraud that they actually did not have This pressure system we have today. So maybe you want could beat up there. But there is a very important thing.

Jörg:   48:51
Yes, but I agree that like this, there are many misplaced incentives and I think we'll touch on that in like, a couple of minutes. Like regarding the open peer review, I think we can all agree that it increases the quality right instead of saying, Oh, yes, at this paper, you have, like, a like at least a structured response. If this is public, um, one thing which I came across in recent years was the p a community in and it's like weaken tied to the peer review thing. We could also tie it to the publishing modern on. And I think this is the interesting concept which is not, has not materialized in our field. Um, from my understanding, however, it, like, was intended to break the power off. The editors know with, like, uh, which should guarantee quality control. But there's, like, a lot of possibility for Jews. Um, are you familiar with this peer community in home, Pablo? And can you maybe talk about the benefits?

Juan Pablo:   50:00
I am not the your you'll have toe. Enlighten me there. I have very little experience. I pull it with these open processes. I am not. I mean, Hess is the only one that yes, I have in county area.

Jörg:   50:13
You know, I think I don't know. I'm not the most knowledgeable about it. I came to me through Carlos 1,000,000 who is like, also working at air back, and he is part of ah p community in ecology. If I'm not mistaken. And the idea is that you have a list off like people who formed this community, it's 100 or 300. And if you would like to publish a paper, you can write to one of thes whether he would endorse that's thing, he endorse your paper and then this person takes the roll off the editor so It's not one editor who makes the decision on which articles appear in a journal or not. But it's like shifting. This, um, distributes the load, which is the nice thing. But it also like, focuses the competences right and like who you choose, that's up to you with It's the big name or whether it's the one who probably has the most knowledge toe, um,

Juan Pablo:   51:14
or your friend.

Jörg:   51:15
Yeah, tools. Well, just that it's just the editor right on. And then this editor. He's looking for people who do a review, and that's all public. And so, in the end, like the editor endorses your paper or not, and that's all you get. So there's no journal attached to it. It's just like an endorsement from, um, I don't have a clue, for example, right? Oh, yeah, that's a really cool thing. The new later. And then you can go to a journal and, like, say, Look, here's the endorsement from paper. Here are these reviews. No, please. Would you accept it for publication in your journal? So the quality control is done by the community and then, like the let's say, the merits, are this then up to you whether you would like Toa say if I publish in nature, that's like a higher quality stamp than the review off renowned professor or professional in your field, Right? Personally, I would be very, very interested to see whether this this peer community in whatever wastewater process or like urban hydrology or urban water even would work. Welcome.

Kris:   52:29
But I would be curious about this system. Network dynamics. Right. Um so if we submit to one journal now, we know what that is. There is, um you can accept that as it er editor as a person and as I understand is that this person puts in, but you cannot accept it and go somewhere else. Yes, Um, so someone you explain this system, um, would make that a bit more free. Or let's say principle, if this works out, there would be more editors to choose from. Yes. Good. Because maybe there are people with another in great standing and very apt to then find reviewers. But then, yeah, the question would be it is just ending up as some sort of Facebook where there are clusters and some clusters should be good and some clusters won't be not, And we all know the good ones want. I mean, yeah, it will make it very hard for somebody external. That has to then this salary Oh, you get a project? Um, because they will not be able to tell whether you are part off the cluster off excellent researchers or whether you're part of the to close us off off mediocre researchers. Yes. So that that's something I would be worried about.

Jörg:   54:07
Yes. And this is like something where we come maybe to the last point, right. The assessment off re Cher's researchers. Because when you mentioned this, I immediately have been thinking about the discussions we had in our group and especially with, like our senior researchers, that if we supervise students right, we have to make sure that they are on the like. We we do everything for them so they could have a like a Korean science. They would like to. And we had, like, a couple of discussions saying, Oh, this is a good Jonah. This is not a good journal that we make sure that our PhD students, they, they they get a good reputation. Andi, like publishing in presumably good journals like is it is easy, easy way for us, maybe as decision makers, or is like a advanced scientists to judge the quality, maybe even to assess people who to apply to work in our lab. Right? So this is like still a very, very open thing. Like how toe assess researchers. Right, Andi, In preparation of this episode, I made a little bit off research and I looked at at a science profile page and I saw that like the H indexes, like the citation off, how many people cite our work are very different, right? Like one of us is low. One of us is high. What does that make the one with the High Re Citation Index? Good scientist.

Juan Pablo:   55:49
I think I'm here. I will make always a distinction Scientists on Academica. That way we need to make when, when you mention the thing about the creditors off helping a students making a good scientific career, I would say what you're making sure is that we have a good academic career. Scientific Kerry isn't affected for where were there probably is actually gets more effective. What the values your culture on them. Andi, The way the systematic, How you train the systematics, it work and how they're good toe the decided. As a result, it's good or bad and so it's really more like the trades off the office off the office. That is the scientific aspect. Everything is how to achieve success in a particular system, which is a Kareena now, I think, and I would like to split this toe where we are aiming at with discussion, like, Is it how we assess scientists or just really, how Karimi had decided today to assess science and then locate their Their resource is based on the

Jörg:   56:54
yes. Ideally, it should be the same, right. That's like what's open signs is about maybe bringing, bringing back, bringing out the good off, the off the off academy.

Juan Pablo:   57:10
I mean, bringing bindle is something that also like, if you look historically, Iwas never good was wars and now I if you go back to the them off cars in Utah on he was no good. It was terrible because they were worth and now so in. I think I carry me has only got better, but it still has a lot off the spices that goes from the very mean which is all about human religion. No, that was scientifically and me and what I see with a lot off pain I have a come to the situation in which I have to accept that the values of academia are not align with scientific bodies. I have to come to split them means they're somehow related. But I cannot think of them us one entity anymore. Although it was educated to think like that. But they e cannot anymore. It just for me extremely a difficult

Jörg:   58:09
Yes, it is. But it's like also, you know, if you educate your PhD student to do everything openly spends, like three months on labeling the maitre data make this called repository public everything you know, And then if it if he applies for tenure, you know, then, uh, old white men, or like whoever decides, you know, they go for metrics, right? No. So when what you typically see is that, for example, in different systems, it's make a huge different people who derive a publication from one year launched 1/2 yearlong master thesis. You know, they are often way ahead off the competitors from European systems where masterpieces are much shorter and there's no way that you get a peer review publication from it in in due time, right? But yes, I am

Juan Pablo:   59:09
in Toto us said the questions. What is what you're able waiting? It is the academic future of this person. Or is the scientific progress off that person? You can be a scientist in any environment. It will be easier or harder. Toe, toe, toe. Act us. One in different environment is clear, but you can be in a current account in one environment. All right, so I'm the question really is like, what is like? I personally always focus more on K if I need toe pass something toe student is the values I believe are positive values in the scientific, a sense right on em. Andi thinks again like, you know, if we are a people that are supposed to be rational in the rational, in the white sense of the economic sense off the war in then a is not doesn't take a lot of time. A lot of coffee is, let's say, to realize that many of the provisions that the economic system has to go against that like actually rational people who lose in that system? A So, Yeah, I'm And I think one should really put clear what is what you're concerned. It is, they say, economic slash academic future of your student? Or is it like okay with the person I'm medicating here? Is he a good scientist? And then you will be a question of how you said scientist No. Then I currently

Kris:   1:0:44
yeah, when I think that distinction is very important. So I think science is a process, uh will happen no matter what may be certain periods of time, it will be slower, certain periods, off time. It will be faster. But But you cannot predict

Juan Pablo:   1:1:04
where that Theo occurred in an

Kris:   1:1:08
academic institute. If it allows for it or equity

Juan Pablo:   1:1:12
happened someone on. I

Kris:   1:1:16
think it's important indeed to distinguish between the institute. Ah, that that claim to partake in scientific process on the process itself. Um, I wanna I wanna bring up thing called Good Hearts Law, which I think is relevant here. And goods flow was basically put out an economist some time ago. It basically says when a measure becomes a target, No, it seizes toe to be a good measure, and I think that's exactly what happened. So it's very before before careers and difficult years were determined. Please don't sign Citation Indexes. The Citation index was a pretty good indicator off the impact of the researcher. No, but now it has become a metric for promotion or for hiring on and sort of people. The highest H and ex ists will be the people that game, the system, and not necessarily as you say, for probably the people that have the actuals scientific prowess are contribution on a related without this, that decisions on a scientific career are made quite early in some sense. So we decide on whether we will hire a postdoc in our group when they're basically just ready to start doing science right there, ready to form later on. My POTUS is of their own experiment, and I got them decide. Let's Avery higher, 20% off. The students based agents for the Postal can continue on their career, and we do that on what basis? Well, I think we use the same measures with same metrics because we have nothing else, but we're trying to evaluate what that person is going to do for the next 30 years or maybe 40 years, which is an incredibly hard problem. Um, Andi

Juan Pablo:   1:3:28
means this inverse. Have the arrow, right? What? You It's OK. If you're I would scientists and your metric should be good. But then as soon as you convert this into a targeted because, like ether measures, I go there. There are good scientists, right? Yes, but no. Where the problems start there.

Jörg:   1:3:41
Yes, but no. We also have, like, creation rule there. Yeah, but we also have a much broader evidence based. Now, you know, if somebody has a code repository with, like, the code is, like, actively developed and, like, downloaded a lot of times writes like that should also count, right? Likely. And that was very difficult toe get to, like, 10 years ago. You know, Now we can We can use this in our assessment, but, um, maybe going for a final statement, you know, like to toe advance the transparency and maybe like you call it, like credibility off science, like, Ah, we could do everything to educate our students. We could, like, improve our own, like, way of working by making our work public, but, um, what couldn't homelessness who Maybe are in ah association or in a funding agency. What could they do to improve this?

Kris:   1:4:50
Well, I think my take home My understanding is that, ah, science is a process is only successful when when it is open. So the idea of open science, it's for me. It's a difficult term because of its not open. Then it cannot be science and some sense, because for me it's very logical that we require science to be open because without it being open that I don't consider it science. I think that's not taken very good. No,

Juan Pablo:   1:5:28
getting in for me is thesis. Is that become a long term? They come basically, but if you're in a position off power, right and I understand many ready on, people have said in the past that you will do it this way. But just try toe him, work over the future, not the past of your institutional. It's like if you get there into your that's a successful situation by following certain behaviour, question yourself. It does the behavior that is actually best for science, for example, and not just for your personal success, and I think it's very hard to toe not perpetrate gaming or winning behaviors system. Right, Because you have this successful bias. One is what made you win. But they are scientists. One should be because we know these things and we are supposed to be trained on question anything. Another life. We should stop and think What do is what we should be doing. And I think it is hard. So maybe since correctly, if you don't understand or less if you don't get in touch with science is supposed to remain anything many years now are improving. Descends like publishes booklet were somehow compile their scientific bodies. They consider core to the institutions. Am I think is good if you read those creation of long A. But you need to know what science is all before you can tell you what is good for site. Decide to see many people in indecision positions that actually not that wears. What is what we're supposed to be doing here? Yes. Besides a publishing paper? This

Jörg:   1:7:21
Yes, yes. No, I, uh I would like to pick on up, pick up on that. I think, like, uh, it's a lot about collaboration science, you know, and it's It's not about the PdF. And I like the term which my PhD supervisor Willie Gloria was using. He was like a referring to the German word. Listen, Shaft, no literally means produce knowledge, right? That's what we have to do and whether it's like publishing it in a good journal. So the engineer in New Zealand or Chile can find what you found out. No, he has a right to know what you what you did. But it's less efficient to have this in PdF's and much more efficient in code in open data and what we've been discussing. The second point is that it's not on Lee. It doesn't make sense to have institute where they're like make super productive people who publish in their own domain without talking to each other, you know? So if you hire somebody, it should also be a good fit. And, like, maybe increased diversity may be bringing new ideas from from different fields like these are also important, important things. Uh huh. Yes. And I think we have been talking like for over an hour again. And it was a pleasure. I would close this a result on open science here, and I would like to thank you, Chris. Thank you. On DA. Thanks. One problem for sharing your insights. Thank you. And you can have a look at the at the web page. If you would like to go deeper, um, get in touch and be open. Theo. Flush to data podcast. Get some extra time. Now with a few minutes of bonus material from Chris York and the guest. Good. You will miss anything here or like was something really wrong

Juan Pablo:   1:9:23
for me, it's a topi. Hard. It's a total. You know why? Because maybe I'm burned with. The thing is, I was have a list here of topics I was mentioning. I wanted to mention it. I long any positive feeling with none of them. Like every time I start talking about any of these topics, I I get a feeling off bitterness. Andi A. For me, it's a very difficult topping. I toe cheer it up. In a way I think I've been They suffering me too, for too long already. And that's why for me, it's hard to see the positive side. But they were discussing licenses. A really civility, public servants, fraud, open science assessment, a best practices on them. I don't. You know when? When I read these items. Well, that's OK. This is good. The feeling I get it's not the positive feeling Like a I don't know if you only someone.

Jörg:   1:10:24
Yeah, I guess you. You mean that? Like, the fact that we have to discuss this at all right is like is already speaking for itself.

Juan Pablo:   1:10:33
Or maybe, you know, it's been more than 20 years. I've been on these and I'm not very positive out. I'm not very personal area because I see people that attached to these values they lose in the system that is seen a science. Academia Still, for many people seen us. Okay, you're a scientist. When you're in academia and most of the say nights off these values that we're discussing here for me our old negative stories are sad story. And

Jörg:   1:11:08
yeah, when I don't know, it's like you. You're not a sad story.

Juan Pablo:   1:11:12
I am a very says. Well, I'm 40 years old, man. I still cannot have ah looked away. Those experiments I want to do.

Jörg:   1:11:21
Yeah, you're not. But with your choices, you will never become a professor these days, right?

Juan Pablo:   1:11:26
Exactly. Yeah, that's what they again. Is that the only way to wear a group? I hope it's not because that but

Jörg:   1:11:33
you can like, get your goals as a kind off like a senior engineer, you know, as you you will have a huge impact. I guess you had already. So in essence, it's like it z not a sad story. I think like what I see Very, very strange. And I was like from this last you project which we had is like that we we we had young guys, right? We come to you and they ask you Hey, I would like to work with you, you know? Okay, then, like what? Your skills like, what's What's your interest? You know? And then, yeah, let's do this. And the second question is, what paper can we right, right, that's that. That's what's what's coming from the young guys. Also even master students, you know, they say, I like your topic. Can we have a publication from this where rewind, right? Like like No, it's not like the role of the scientists to produce Pdf's right. And I think that, like from when when when we started like ah Ph. D group. Now we had discussions like why you're doing a PhD. And then people said, Well, I want I want to know I want to educate myself, right? I wanna know how. Ah, how to program this thing. Or like, it was more about about the processes and less about the products I have. I have similar

Kris:   1:12:59
observations. One was a college just recently. So is preparing a technical report, which just very mundane that say it's important. But it's not a paper, you know. There is no scientific questions.

Juan Pablo:   1:13:16
Thank you kind of force are dead, crease. Like who does it anymore?

Kris:   1:13:20
Yeah, but the task the task I'm working on giving an overview of something. I'm a good rating chart number. Collective security now. Yeah. Yeah. So that's what that's what the task is. And then after two weeks for that Oh, this is going nicely. Um, we could probably get a paper out of this. So luckily, I had some clout, and I could think No. If you wanna end of paper of this, you do it. But that's this is No, it's just not where this is going, you know? And so I'm happy to have that weight that I'm able to say things like that but for young people is not so easy.

Juan Pablo:   1:14:04
But you see, in principle, if one would dig deeper insight like why you want to make a paper If it and I find a person, say why I would like to make a beer? Because look, what we're doing here could be useful for narrow somebody if that is a vacation for you, and I'm totally, probably already I mean a person, Caesar. But that's send them together, at least in my experience. No, what you see, like why they want the Paris of papers. Because if they perceive us, you need to do

Jörg:   1:14:32
yes, exactly. But this getting. But I don't want to play that

Kris:   1:14:35
person. It ties together with some institutional incentives. It was just a month ago. There was some email a car would not admit bleeding. We're not reaching our targets. The what targets are number of publications, what sai so and I'm on the good side because I publish more notice to average ordinary person. Um, because I think relatively speaking, with pressures at air work for publication, are a bit higher than here. But now there's a lot off other people that so I compare it probably air work, maybe 10 15 years ago. It's where our analysts and something that increasing the pressure for publication now,

Jörg:   1:15:24
which is totally stupid like, which is totally stupid. Yeah, I not such so many discussions, right? Like, yeah, like, ah, when I did my PhD, I can't remember. It was like, Ah e th PhD 15,000 or so And now my, my first PhD student was 21,000 or so like because they count the PhD said e th you know, and that's from 18 fifties or 18. I don't know when ET Age was founded. You know, it's 15,000 and from 2005 to 2010 it's like 6000 more. Come on, you know, And then, like in our own environment, you know, like it was, Yeah, you have 11 paper accepted in a period journal and two submitted was the standard. Now it's two accepted and once, and that's like it's fueling that if putting fuel into the fire right, but it's like it should be exactly

Juan Pablo:   1:16:22
they represent, just ask around. I've done this several times, like Why as white, Why they do things like why you publish? What? Why are we supposed about? What's they they recently hand Yes on. And it is seldom when you had a satisfying like this. What do you say, Mason like? Yeah, because it's the only way we have to share. No, which is not true. But I am Yes, but I think it is seldom dancer because we want to share with was going to contribute records, right? Yes.

Jörg:   1:16:55
But back in the day, you know, that was like what really was saying like, it doesn't make sense to have ah, 300 page monograph off your PhD in Ukrainian or in in some Chilean Andean dialect, you know, because it's lost if you make it three papers, you know, you presented conferences, it's it's there, and people confined it. Now with these technical reports, you you put them on a repository and people confined it so personally in terms off creating the knowledge this there should be much less pressure.

Juan Pablo:   1:17:25
It should be a lot easier, right?

Jörg:   1:17:26
Yes. And if you're technical report has been downloaded 200 times, you know, that should also be considered some

Juan Pablo:   1:17:34
house by a robot that I program. Yeah, that's any when you're supporting the missions, right?

Jörg:   1:17:43
I was going to get to that. We didn't discuss

Kris:   1:17:46
that. That's maybe one thing, if now for you with the thing that you shared your before we make the call. You said Now new incentives to science has to be open on the after release open data and open coat. Does that mean that a number off people will then start producing garbage data and garbage? Cold, I would say

Juan Pablo:   1:18:12
it isn't likely is becomes a measure. That's what will happen. And that's why I'm completely against, like, weaponizing these things because it's exactly what will happen as soon as you say, Oh, you're a good science scientists. If you have so many reports it, you will start seeing just crap created. It's gonna be like that. You put more money in the field, the more crab you will see

Kris:   1:18:35
to some. And I understand that in sex Silicon Valley like you get help, can does it commit, uh, being used as a metric? Not not in academia. Put it in the Google on my phone start ups. I think if a certain degree it's already happening.

Juan Pablo:   1:19:01
I know way This may not be some reasonable to you, but I believe the only way other people cannot know how you measure them is I keep in the process subject. My off course will agrees. Have ideas a permanent off much production Like what happened when you have, like, huge volumes and also have the thing. Okay, this problem off biases dama by a personal bias that you may have but and I think there is at the end of a you can also produce a like If you have so many so many measurements, right, it's very hard to track which one. We're so very knowing a process toe Ever wait a in something that I think that will be a little bit more interact with Boyd gaming the system. But nowadays we have, like Oneto one revelations like it was impact factor for away. Now it's becoming older. A measurements on the is like clerics. We have a ways of measuring people the way a clerk will do it, right? Yes. And that is a sign.

Jörg:   1:20:08
Yes, And I had a conversation on Twitter about this, you know where, like how that's ah about hiring people, right? Like if you get 10 applications or five as we get maybe 50 you know, you can sort out the ones you can look at their literature least and, like, read the paper which they published, you know, get like, how does this person think? How can you cannae right. But if you get 10,000 applications right, that's impossible. Impossible. Andi going digital, right? It's like if people don't have to write a letter physically, but they can produce a pdf and send it 200 emails. You know, that's the scalability. That's Ah, it's a real problem, you know?

Juan Pablo:   1:20:54
So it's every prior. So you have these screenings basic, right, that will reduce this number. So reasonable amount,

Jörg:   1:21:00
yes. Or everybody has to pay 20 bucks to apply, you know, or like, make some heard, like write a letter, you know, and then, yeah, it can't be time right to physical letter with pen and paper that people from Africa can do, or people are not so well off, right? But otherwise the rich guy sends out 100 emails and the poor guy can't. So it's it's really difficult. And I think one of us. One thing we we have. Sorry, but

Juan Pablo:   1:21:29
just like you believe that randomness fair or is unfair because you have people is paralyzed years above your randomness fair on the miss over there.

Jörg:   1:21:43
I think it was fair, but maybe not efficient.

Juan Pablo:   1:21:48
I mean, so the manifestation you let's say your manpower can process in the time of their life. 20 applications. You got 1000. So you need to get rid of this man 80. So for me, what I have seen at least like any process that you need to make public, and then the guys applying will will produce certain advantages with advantages for some were already pre select a group, right? And so I was wondering, Well, maybe the most fertile. We just randomly 20 off those a aggregate. Definitely.

Jörg:   1:22:25
It's like, uh, but it's it's it's not efficient because you lose the super. Then the next Einstein, you know, there's a high chance that you lose him.

Kris:   1:22:36
But the Armstrong wasn't working at the university.

Juan Pablo:   1:22:39
Oh, hey, hey. Was very bad. It's the comments off. His world was horrible. Yes, He

Kris:   1:22:48
was not satisfying any metric. Yes. Hey, wouldn't today either?

Jörg:   1:22:56
Yes. So I think What? You what you mentioned compound like the why is really good. And I remember I was reading up the story of the laser because somebody mentioned, you know, the laser was it was that was not like on purpose invented, you know, like the blue brain like we are. We need to computerize our mental capacity capacity. You know, it was like just some guy in the basement trying to focus light in the Khalis. Why? Why are you doing this like no. Yeah, may weaken. Make something which raises letters from a badly written typewriter document, you know, So you So he didn't have any clue what what was behind it. You know, it was just the pure signed. Oh, I can energize these atoms, and this will be really cool, you know? And then there's no way to predict now in the fifties or sixties when they did it. Like what? The impact off a laser waas right with from CDs to music production to like, highly precise. But whatever you know, it's like it's there was no way to predict or to assess people in ah, in a objective former, because your frame of reference is like whatever

Juan Pablo:   1:24:09
1,000,000 Einstein with the general theory of relativity applying toe snf embassy oniy He doesn't get it, man. No, because it would say, what is the relevance of your results in the next five years? Like, who knows, e. I mean, of course we'll be able to run, maybe for for theoretical physics, what is a but it's it's impossible a any in this knowledge, fabrication means that we don't know what we're going on, right? Course we may know how we're gonna try to do it right? This I think on this one, we can request about a scientist like, can you explain what you plan to do but don't try to justify, You know, the values in the base on? Yeah, I don't know. A inventions, patents, and whatever papers, you know, we're gonna probably so many papers off these and yeah, it's a it's it is

Kris:   1:25:07
better. It's a difficult came to after funding and play, at least in your car. Have to play that game and then get the money and then somehow managed to do something. You do? Yeah, Exactly. Yeah. Um it's not impossible, but I have the same feelings I just don't yesterday, I think.

Juan Pablo:   1:25:34
But again, that's what I guess. These executive that appointees, Doesn't the scientists of you get extremely border by this thing? Like why we all rational people leading a Russia Because all the systems are basically executed by most of the time by scientists. Right. And why are we all accepted? This is the thing. Why are we right in application? We all know are basically bullshitting to get high rate off acceptance and not actually a true description of what you're gonna do. Why I win the system. All right? Yeah,

Jörg:   1:26:07
I think Eric, there's a guy

Kris:   1:26:09
called Eric Weinstein made some points to that. He used to be an M I t. I think, and he left and he came up with a proposal for ah, dearly that is supposed to unify physics of unified general relativity and quantum physics. So it's a proposal. He's very clear about that. There's no evidence for it, but he basically said I'm mighty, didn't allow me, didn't didn't allow me to even spend time on that.

Jörg:   1:26:41
So he's on his in the patent office now, like

Kris:   1:26:45
I think, to certain degree, he probably created a position for himself like that where he's basically allowed to spend time.

Jörg:   1:26:57
Yeah, I mean, if you ask about the why, I guess it's about like, uh, because that's what we do, right? It's like it's like, um, professional know why are we still in the system, right? It's probably like it's like a like a professional football player, right? I mean, these are these kids who will play with the ball like 24 7 you know, it's all about the ball, you know, it's and then they become professional. It's now and then they have to not dribble and, like, do solo all the time. But they have to play with the team. They have to follow certain rules, right to be successful, whatever that is. And it's still about the fun, right? The guy with a laser and like when we write our oh, our stuff for, like think about our ideas, you know that's the system makes it possible to do so. And I have done when we prepared for this episode like I I was asking thinking about Chris like brainstorming glue. How was it in Godson cows? Time right for these people, evaluated was KAOS actually evaluated. I think from my understanding was that there was the king and he had five scientists. No one is a new whisk it coming up, you know, like it was. It was believing the mentor, that it was a good guy, you know? And then that's it.

Juan Pablo:   1:28:24
It'll mean Beyer that muscle,

Jörg:   1:28:27
It was young well, but also like personal stuff, right? Maybe. I mean, girls was recognised. His brilliance was recognized, you know. So it's I might

Kris:   1:28:40
I think, to dio to the country leaders at the time it was a magic for the country. Right? So we have cows, we have rehab. Really? Was is like advertisement for the country, probably. And then I imagine somebody that cause you call one time per year to the parade to the dinner and the King shows off with This is our most fairly and scientists. Yes. I think that's the trade that these people were supposed to make.

Jörg:   1:29:20
Yes, And that's what we're doing also right. Once per year, we write our proposal and I had a had a quite good commentary about the physics community, but they say, Oh, we all open. You know, we are like all about archives and lycopene code. And but if you apply for funding you, that's that's where the but shit hits the fan. You know, like, because then somebody has to give you money or not, you know, And then it's like a split second decision because they're 200 proposals to be read, write, and then back. We are

Kris:   1:29:55
somebody, like, every hard saying, Well, if I have to check archive now, too, they have plenty of work checking water research on environmental

Jörg:   1:30:04
side. Yes. Knowledge, yes. But then it's the random argument. You know, you you pick 10. Because that's what you can manage with, Like your three kids running around and like, Yeah,

Kris:   1:30:15
yeah, it's in. Yeah, I I I can see the sentiment from Pablo. Just just a rare unlike Dennis, the Beastie student Seo, my first paper published Fantastic. I hope to publish many more. And I immediately thought why? I mean, no, I mean that this this, uh so how to say so young and academic career. And it's already brainwashed authority. It's already incorporated this idea that it's about the big ears.

Juan Pablo:   1:30:59
Yes, Well, you remember you're a priest, meaning they close toe the robotics community. They're really committees. It's crazy, you say is there is no respect. That is like, I don't think me at least many of the colleagues I have, like they wouldn't even not know why they actually right enough. The year. It just what they do is what we do. And we need to produce one per month. But a papa. Papa, Papa, Not your Children in it. Yeah, okay. It's a it's basically like sometimes, I mean, I remember this was from other bunnies. This is what I get paid for

Kris:   1:31:38
a company.

Jörg:   1:31:39
Yes, but I think what you mentioned, you know that it was worse. Worse in the guards times, right?

Juan Pablo:   1:31:46
There was no system there. Yeah,

Jörg:   1:31:48
or like total total randomness or whatever. You know, I think that's like maybe a hopeful thing,

Juan Pablo:   1:31:55
huh? I am convinced a kareena now is much better than the time. Because of the time there was a lot of personal power. I mean, we see from history on Lee. The scientists were lucky, or they have the right contact. There have lost people that we don't know about way can single out, you know, like the these super geniuses Yeah, probably. There were many more that we just don't see.

Jörg:   1:32:17
There were many more. Yes. We're like farmers, sons or whatever Who did not have the Yes. And maybe with this open systems, you know, the traceability is also a possibility. You know that you can or you have to be aware that you stand scrutiny or you can be tested right? If things out, you know, it's not need it necessarily. But like, if you really fuck it up, you know, Then people will find it. You know, that's also something which gives me hope.

Kris:   1:32:51
The meat is the process. The process going?

Jörg:   1:32:54
Yes, Yes. Retracting Watch. That's what we forgot. Good. Thanks, everybody.

Juan Pablo:   1:33:01
Thank you. NYT. Yeah. Yeah.

Jörg:   1:33:04
Good night. Stay healthy.