Data Science at Home

There is a connection between gradient descent based optimizers and the dynamics of damped harmonic oscillators. What does that mean? We now have a better theory for optimization algorithms. In this episode I explain how all this works. All the formulas I mention in the episode can be found in the post The physics of optimization algorithms Enjoy the show.

How are differential equations related to neural networks? What are the benefits of re-thinking neural network as a differential equation engine? In this episode we explain all this and we provide some material that is worth learning. Enjoy the show! Residual Block References [1] K. He, et al., “Deep Residual Learning for Image Recognition”, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 770-778, 2016 [2] S. Hochreiter, et al., “Long short-term memory”, Neural Computation 9(8), pages 1735-1780, 1997. [3] Q. Liao, et al.,”Bridging the gaps between residual learning, recurrent neural networks and visual cortex”, arXiv preprint, arXiv:1604.03640, 2016. [4] Y. Lu, et al., “Beyond Finite Layer Neural Networks: Bridging Deep Architectures and Numerical Differential Equation”, Proceedings of the 35th International Conference on Machine Learning (ICML), Stockholm, Sweden, 2018. [5] T. Q. Chen, et al., ” Neural Ordinary Differential Equations”, Advances in Neural Information Processing Systems 31, pages 6571-6583}, 2018

Since the beginning of AI in the 1950s and until the 1980s, symbolic AI approaches have dominated the field. These approaches, also known as expert systems, used mathematical symbols to represent objects and the relationship between them, in order to depict the extensive knowledge bases built by humans. The opposite of the symbolic AI paradigm is named connectionism, which is behind the machine learning approaches of today

The successes that deep learning systems have achieved in the last decade in all kinds of domains are unquestionable. Self-driving cars, skin cancer diagnostics, movie and song recommendations, language translation, automatic video surveillance, digital assistants represent just a few examples of the ongoing revolution that affects or is going to disrupt soon our everyday life. But all that glitters is not gold… Read the full post on the Amethix Technologies blog

In this episode I speak about how important reproducible machine learning pipelines are. When you are collaborating with diverse teams, several tasks will be distributed among different individuals. Everyone will have good reasons to change parts of your pipeline, leading to confusion and definitely a number of options that soon explode. In all those cases, tracking data and code is extremely helpful to build models that are reproducible anytime, anywhere. Listen to the podcast and learn how.

Have you ever wanted to get an estimate of the uncertainty of your neural network? Clearly Bayesian modelling provides a solid framework to estimate uncertainty by design. However, there are many realistic cases in which Bayesian sampling is not really an option and ensemble models can play a role. In this episode I describe a simple yet effective way to estimate uncertainty, without changing your neural network’s architecture nor your machine learning pipeline at all. The post with mathematical background and sample source code is published here.

The success of a machine learning model depends on several factors and events. True generalization to data that the model has never seen before is more a chimera than a reality. But under specific conditions a well trained machine learning model can generalize well and perform with testing accuracy that is similar to the one performed during training. In this episode I explain when and why machine learning models fail from training to testing datasets.

In this episode I am completing the explanation about the integration fitchain-oceanprotocol that allows secure on-premise compute to operate in the decentralized data marketplace designed by Ocean Protocol. As mentioned in the show, this is a picture that provides a 10000-feet view of the integration. I hope you enjoy the show!

In this episode I briefly explain how two massive technologies have been merged in 2018 (work in progress :) - one providing secure machine learning on isolated data, the other implementing a decentralized data marketplace. In this episode I explain: How do we make machine learning decentralized and secure? How can data owners keep their data private? How can we benefit from blockchain technology for AI and machine learning? I hope you enjoy the show! References fitchain.io decentralized machine learnin Ocean protocol decentralized data marketplace

It's always good to put in perspective all the findings in AI, in order to clear some of the most common misunderstandings and promises. In this episode I make a list of some of the most misleading statements about what artificial intelligence can achieve in the near future.

In this episode - which I advise to consume at night, in a quite place - I speak about private machine learning and blockchain, while I sip a cup of coffee in my home office. There are several reasons why I believe we should start thinking about private machine learning... It doesn't really matter what approach becomes successful and gets adopted, as long as it makes private machine learning possible. If people own their data, they should also own the by-product of such data. Decentralized machine learning makes this scenario possible.

Today I am having a conversation with Filip Piękniewski, researcher working on computer vision and AI at Koh Young Research America. His adventure with AI started in the 90s and since then a long list of experiences at the intersection of computer science and physics, led him to the conclusion that deep learning might not be sufficient nor appropriate to solve the problem of intelligence, specifically artificial intelligence. I read some of his publications and got familiar with some of his ideas. Honestly, I have been attracted by the fact that Filip does not buy the hype around AI and deep learning in particular. He doesn’t seem to share the vision of folks like Elon Musk who claimed that we are going to see an exponential improvement in self driving cars among other things (he actually said that before a Tesla drove over a pedestrian).

In this episode I continue the conversation from the previous one, about failing machine learning models. When data scientists have access to the distributions of training and testing datasets it becomes relatively easy to assess if a model will perform equally on both datasets. What happens with private datasets, where no access to the data can be granted? At fitchain we might have an answer to this fundamental problem.

The success of a machine learning model depends on several factors and events. True generalization to data that the model has never seen before is more a chimera than a reality. But under specific conditions a well trained machine learning model can generalize well and perform with testing accuracy that is similar to the one performed during training. In this episode I explain when and why machine learning models fail from training to testing datasets.

In this episode I don't talk about data. In fact, I talk about metadata. While many machine learning models rely on certain amounts of data eg. text, images, audio and video, it has been proved how powerful is the signal carried by metadata, that is all data that is invisible to the end user. Behind a tweet of 140 characters there are more than 140 fields of data that draw a much more detailed profile of the sender and the content she is producing... without ever considering the tweet itself. References You are your Metadata: Identification and Obfuscation of Social Media Users using Metadata Information https://www.ucl.ac.uk/~ucfamus/papers/icwsm18.pdf

Today’s episode is about text analysis with python. Python is the de facto standard in machine learning. A large community, a generous choice in the set of libraries, at the price of less performant tasks, sometimes. But overall a decent language for typical data science tasks. I am with Rebecca Bilbro, co-author of Applied Text Analysis with Python, with Benjamin Bengfort and Tony Ojeda. We speak about the evolution of applied text analysis, tools and pipelines, chatbots.

Attacking deep learning models Compromising AI for fun and profit Deep learning models have shown very promising results in computer vision and sound recognition. As more and more deep learning based systems get integrated in disparate domains, they will keep affecting the life of people. Autonomous vehicles, medical imaging and banking applications, surveillance cameras and drones, digital assistants, are only a few real applications where deep learning plays a fundamental role. A malfunction in any of these applications will affect the quality of such integrated systems and compromise the security of the individuals who directly or indirectly use them. In this episode, we explain how machine learning models can be attacked and what we can do to protect intelligent systems from being compromised.

Today’s episode will be about deep learning and reasoning. There has been a lot of discussion about the effectiveness of deep learning models and their capability to generalize, not only across domains but also on data that such models have never seen. But there is a research group from the Department of Computer Science, Duke University that seems to be on something with deep learning and interpretability in computer vision. References Prediction Analysis Lab Duke University https://users.cs.duke.edu/~cynthia/lab.html This looks like that: deep learning for interpretable image recognition https://arxiv.org/abs/1806.10574

Today’s episode will be about deep learning and compression of data, and in particular compressing images. We all know how important compressing data is, reducing the size of digital objects without affecting the quality. As a very general rule, the more one compresses an image the lower the quality, due to a number of factors like bitrate, quantization error, etcetera. I am glad to be here with Tong Chen, researcher at the School of electronic Science and Engineering of Nanjing University, China. Tong developed a deep learning based compression algorithm for images, that seems to improve over state of the art approaches like BPG, JPEG2000 and JPEG. Reference Deep Image Compression via End-to-End Learning - Haojie Liu, Tong Chen, Qiu Shen, Tao Yue, and Zhan Ma School of Electronic Science and Engineering, Nanjing University, Jiangsu, China

In this episode I explain the differences between L1 and L2 regularization that you can find in function minimization in basically any machine learning model.

In the second part of this episode I am interviewing Johannes Castner from CollectiWise, a platform for collective intelligence. I am moving the conversation towards the more practical aspects of the project, asking about the centralised AGI and blockchain components that are essential part of the platform. References Opencog.org Thaler, Richard H., Sunstein, Cass R. and Balz, John P. (April 2, 2010). "Choice Architecture". doi:10.2139/ssrn.1583509. SSRN 1583509 Teschner, F., Rothschild, D. & Gimpel, H. Group Decis Negot (2017) 26: 953. https://doi.org/10.1007/s10726-017-9531-0 Firas Khatib, Frank DiMaio, Foldit Contenders Group, Foldit Void Crushers Group, Seth Cooper, Maciej Kazmierczyk, Miroslaw Gilski, Szymon Krzywda, Helena Zabranska, Iva Pichova, James Thompson, Zoran Popović, Mariusz Jaskolski & David Baker, Crystal structure of a monomeric retroviral protease solved by protein folding game players, Nature Structural & Molecular Biology volume18, pages1175–1177 (2011) Rosenthal, Franz; Dawood, Nessim Yosef David (1969). The Muqaddimah : an introduction to history ; in three volumes. 1. Princeton University Press. ISBN 0-691-01754-9. Kevin J. Boudreau and Karim R. Lakhani, Using the Crowd as an Innovation Partner, April 2013. Sam Bowles, The Moral Economy: Why Good Incentives are No Substitute for Good Citizens. Amartya K. Sen, Rational Fools: A Critique of the Behavioral Foundations of Economic Theory, Philosophy & Public Affairs, Vol. 6, No. 4 (Summer, 1977), pp. 317-344, Published by: Wiley, Stable URL: http://www.jstor.org/stable/2264946

This is the first part of the amazing episode with Johannes Castner, CEO and founder of CollectiWise. Johannes is finishing his PhD in Sustainable Development from Columbia University in New York City, and he is building a platform for collective intelligence. Today we talk about artificial general intelligence and wisdom. All references and shownotes will be published after the next episode. Enjoy and stay tuned!

Predicting the weather is one of the most challenging tasks in machine learning due to the fact that physical phenomena are dynamic and riche of events. Moreover, most of traditional approaches to climate forecast are computationally prohibitive. It seems that a joint research between the Earth System Science at the University of California, Irvine and the faculty of Physics at LMU Munich has an interesting improvement on the scalability and accuracy of climate predictive modeling. The solution is... superparameterization and deep learning. References Could Machine Learning Break the Convection Parameterization Deadlock? Gentine, M. Pritchard, S. Rasp, G. Reinaudi, and G. Yacalis Earth and Environmental Engineering, Columbia University, New York, NY, USA, Earth System Science, University of California, Irvine, CA, USA, Faculty of Physics, LMU Munich, Munich, Germany

Humans seem to have reached a cross-point, where they are asked to choose between functionality and privacy. But not both. Not both at all. No data, no service. That’s what companies building personal finance services say. The same applies to marketing companies, social media companies, search engine companies, and healthcare institutions. In this episode I speak about the reasons to aggregate data for precision medicine, the consequences of such strategies and how can researchers and organizations provide services to individuals while respecting their privacy.

Attacking deep learning models Compromising AI for fun and profit Deep learning models have shown very promising results in computer vision and sound recognition. As more and more deep learning based systems get integrated in disparate domains, they will keep affecting the life of people. Autonomous vehicles, medical imaging and banking applications, surveillance cameras and drones, digital assistants, are only a few real applications where deep learning plays a fundamental role. A malfunction in any of these applications will affect the quality of such integrated systems and compromise the security of the individuals who directly or indirectly use them. In this episode, we explain how machine learning models can be attacked and what we can do to protect intelligent systems from being compromised.

Today I am having a conversation with Filip Piękniewski, researcher working on computer vision and AI at Koh Young Research America. His adventure with AI started in the 90s and since then a long list of experiences at the intersection of computer science and physics, led him to the conclusion that deep learning might not be sufficient nor appropriate to solve the problem of intelligence, specifically artificial intelligence. I read some of his publications and got familiar with some of his ideas. Honestly, I have been attracted by the fact that Filip does not buy the hype around AI and deep learning in particular. He doesn’t seem to share the vision of folks like Elon Musk who claimed that we are going to see an exponential improvement in self driving cars among other things (he actually said that before a Tesla drove over a pedestrian).

In the attempt of democratizing machine learning, data scientists should have the possibility to train their models on data they do not necessarily own, nor see. A model that is privately trained should be verified and uniquely identified across its entire life cycle, from its random initialization to setting the optimal values of its parameters. How does blockchain allow all this? Fitchain is the decentralized machine learning platform that provides models an identity and a certification of their training procedure, the proof-of-train

I know, I have been away too long without publishing much in the last 3 months. But, there's a reason for that. I have been building a platform that combines machine learning with blockchain technology. Let me introduce you to fitchain and tell you more in this episode. If you want to collaborate on the project or just think it's interesting, drop me a line on the contact page at fitchain.io

Cross-posting from Cryptoradio.io Overview Francesco Gadaleta introduces Fitchain, a decentralized machine learning platform that combines blockchain technology and AI to solve the data manipulation problem in restrictive environments such as healthcare or financial institutions.Francesco Gadaleta is the founder of Fitchain.io and senior advisor to Abe AI. Fitchain is a platform that officially started in October 2017, which allows data scientists to write machine learning models on data they cannot see and access due to restrictions imposed in healthcare or financial environments. In the Fitchain platform, there are two actors, the data owner and the data scientist. They both run the Fitchain POD, which orchestrates the relationship between these two sides. The idea behind Fitchain is summarized in the thesis “do not move the data, move the model – bring the model where the data is stored.” The Fitchain team has also coined a new term called “proof of train” – a way to guarantee that the model is truly trained at the organization, and that it becomes traceable on the blockchain. To develop the complex technological aspects of the platform, Fitchain has partnered up with BigChainDB, the project we have recently featured on Crypto Radio. Roadmap Fitchain team is currently validating the assumptions and increasing the security of the platform. In the next few months, they will extend the portfolio of machine learning libraries and are planning to move from a B2B product towards a Fitchain for consumers. By June 2018 they plan to start the Internet of PODs. They will also design the Fitchain token – FitCoin, which will be a utility token to enable operating on the Fitchain platform.

Data is a complex topic, not only related to machine learning algorithms, but also and especially to privacy and security of individuals, the same individuals who create such data just by using the many mobile apps and services that characterize their digital life. In this episode I am together with B.J.n Mendelson, author of “Social Media is Bullshit” from St. Martin’s Press and world-renowned speaker on issues involving the myths and realities involving today’s Internet platforms. B.J. has a new a book about privacy and sent me a free copy of "Privacy, and how to get it back" that I read in just one day. That was enough to realise how much we have in common when it comes to data and data collection.

Despite what researchers claim about genetic evolution, in this episode we give a realistic view of the field.

In order to succeed with artificial intelligence, it is better to know how to fail first. It is easier than you think. Here are 9 easy steps to fail your AI startup.

The enthusiasm for artificial intelligence is raising some concerns especially with respect to some ventured conclusions about what AI can really do and what its direct descendent, artificial general intelligence would be capable of doing in the immediate future. From stealing jobs, to exterminating the entire human race, the creativity (of some) seems to have no limits. In this episode I make sure that everyone comes back to reality - which might sound less exciting than Hollywood but definitely more... real.

In the aftermath of the Barclays Accelerator, powered by Techstars experience, one of the most innovative and influential startup accelerators in the world, I’d like to give back to the community lessons learned, including the need for confidence, soft-skills, and efficiency, to be applied to startups that deal with artificial intelligence and data science. In this episode I also share some thoughts about the culture of fireflies in modern and dynamic organisations.

In this episode I speak about Deep Learning technology applied to Alzheimer disorder prediction. I had a great chat with Saman Sarraf, machine learning engineer at Konica Minolta, former lab manager at the Rotman Research Institute at Baycrest, University of Toronto and author of DeepAD: Alzheimer′ s Disease Classification via Deep Convolutional Neural Networks using MRI and fMRI. I hope you enjoy the show.

In this episode, I speak about the requirements and the skills to become data scientist and join an amazing community that is changing the world with data analyticsa

In machine learning and data science in general it is very common to deal at some point with imbalanced datasets and class distributions. This is the typical case where the number of observations that belong to one class is significantly lower than those belonging to the other classes. Actually this happens all the time, in several domains, from finance, to healthcare to social media, just to name a few I have personally worked with. Think about a bank detecting fraudulent transactions among millions or billions of daily operations, or equivalently in healthcare for the identification of rare disorders. In genetics but also with clinical lab tests this is a normal scenario, in which, fortunately there are very few patients affected by a disorder and therefore very few cases wrt the large pool of healthy patients (or not affected). There is no algorithm that can take into account the class distribution or the amount of observations in each class, if it is not explicitly designed to handle such situations. In this episode I speak about some effective techniques to handle imbalanced datasets, advising the right method, or the most appropriate one to the right dataset or problem. In this episode I explain how to deal with such common and challenging scenarios.

Ensemble methods have been designed to improve the performance of the single model, when the single model is not very accurate. According to the general definition of ensembling, it consists in building a number of single classifiers and then combining or aggregating their predictions into one classifier that is usually stronger than the single one. The key idea behind ensembling is that some models will do well when they model certain aspects of the data while others will do well in modelling other aspects. In this episode I show with a numeric example why and when ensemble methods work.

Continuing the discussion of the last two episodes, there is one more aspect of deep learning that I would love to consider and therefore left as a full episode, that is parallelising and distributing deep learning on relatively large clusters. As a matter of fact, computing architectures are changing in a way that is encouraging parallelism more than ever before. And deep learning is no exception and despite the greatest improvements with commodity GPUs - graphical processing units, when it comes to speed, there is still room for improvement. Together with the last two episodes, this one completes the picture of deep learning at scale. Indeed, as I mentioned in the previous episode, How to master optimisation in deep learning, the function optimizer is the horsepower of deep learning and neural networks in general. A slow and inaccurate optimisation method leads to networks that slowly converge to unreliable results. In another episode titled “Additional strategies for optimizing deeplearning” I explained some ways to improve function minimisation and model tuning in order to get better parameters in less time. So feel free to listen to these episodes again, share them with your friends, even re-broadcast or download for your commute. While the methods that I have explained so far represent a good starting point for prototyping a network, when you need to switch to production environments or take advantage of the most recent and advanced hardware capabilities of your GPU, well... in all those cases, you would like to do something more.

In the last episode How to master optimisation in deep learning I explained some of the most challenging tasks of deep learning and some methodologies and algorithms to improve the speed of convergence of a minimisation method for deep learning. I explored the family of gradient descent methods - even though not exhaustively - giving a list of approaches that deep learning researchers are considering for different scenarios. Every method has its own benefits and drawbacks, pretty much depending on the type of data, and data sparsity. But there is one method that seems to be, at least empirically, the best approach so far. Feel free to listen to the previous episode, share it, re-broadcast or just download for your commute. In this episode I would like to continue that conversation about some additional strategies for optimising gradient descent in deep learning and introduce you to some tricks that might come useful when your neural network stops learning from data or when the learning process becomes so slow that it really seems it reached a plateau even by feeding in fresh data.

The secret behind deep learning is not really a secret. It is function optimisation. What a neural network essentially does, is optimising a function. In this episode I illustrate a number of optimisation methods and explain which one is the best and why.

Over the past few years, neural networks have re-emerged as powerful machine-learning models, reaching state-of-the-art results in several fields like image recognition and speech processing. More recently, neural network models started to be applied also to textual data in order to deal with natural language, and there too with promising results. In this episode I explain why is deep learning performing the way it does, and what are some of the most tedious causes of failure.

Artificial Intelligence allow machines to learn patterns from data. The way humans learn however is different and more efficient. With Lifelong Machine Learning, machines can learn the way human beings do, faster, and more efficiently

Talking about security of communication and privacy is never enough, especially when political instabilities are driving leaders towards decisions that will affect people on a global scale

We strongly believe 2017 will be a very interesting year for data science and artificial intelligence. Let me tell you what I expect and why.

Is the market really predictable? How do stock prices increase? What is their dynamics? Here is what I think about the magics and the reality of predictions applied to markets and the stock exchange.

Why the job of the data scientist can disappear soon. What is required by a data scientist to survive inflation.

Data science is making the difference also in fraud detection. In this episode I have a conversation with an expert in the field, Engineer Eyad Sibai, who works at iZettle, a fraud detection company

Extracting knowledge from large datasets with large number of variables is always tricky. Dimensionality reduction helps in analyzing high dimensional data, still maintaining most of the information hidden behind complexity. Here are some methods that you must try before further analysis (Part 1).

How would you perform accurate classification on a very large dataset by just looking at a sample of it