Jacob Cho's Data Science

PyData DC 2018 Recreating, Understanding, and Visualizing FiveThirtyEight's Elections Forecast Think about forecasting results based on polls, how to visualize your results, and explain them. Now apply that to a flashy, real world example: the 2018 midterms. The core tutorial covers sampling, simulations, statistics, interactive visualization, and communicating results. Not to fear: this tutorial is ultra "hands-on" with every concept being immediately applied with NumPy, SciPy, and Pandas. Slides - https://github.com/josephofiowa/pydat... Code - https://github.com/josephofiowa/pydat... === www.pydata.org PyData is an educational program of NumFOCUS, a 501(c)3 non-profit organization in the United States. PyData provides a forum for the international community of users and developers of data analysis tools to share ideas and learn from each other. The global PyData network promotes discussion of best practices, new approaches, and emerging technologies for da...

PyData DC 2018 Most of your time is going to involve processing/cleaning/munging data. How do you know your data is clean? Sometimes you know what you need beforehand, but other times you don't. We'll cover the basics of looking at your data and getting started with the Pandas Python library, and then focus on how to "tidy" and reshape data. We'll finish with applying customized processing functions on our data. === www.pydata.org PyData is an educational program of NumFOCUS, a 501(c)3 non-profit organization in the United States. PyData provides a forum for the international community of users and developers of data analysis tools to share ideas and learn from each other. The global PyData network promotes discussion of best practices, new approaches, and emerging technologies for data management, processing, analytics, and visualization. PyData communities approach data science using many languages, including (but not limited to) Python, Julia, and R. ...

PyData DC 2018 We will discuss and demonstrate some of the exciting features that have been added to version 3 of the plotly Python visualization library. These include ipywidgets integration for building ad-hoc dashboards in the Jupyter Notebook, support for exporting publication quality static images, Latex integration, and more. We will conclude with glimpse into the future direction of the project. Slides - https://drive.google.com/open?id=1V8P... === www.pydata.org PyData is an educational program of NumFOCUS, a 501(c)3 non-profit organization in the United States. PyData provides a forum for the international community of users and developers of data analysis tools to share ideas and learn from each other. The global PyData network promotes discussion of best practices, new approaches, and emerging technologies for data management, processing, analytics, and visualization. PyData communities approach data science using many languages, including (but not limited to) Pyt...

Pandas 기초 - 시계열 데이터(Time Series Data) 분석을 위한 판다스 Expanding and Rolling 이해하기 * 참고링크 : https://pandas.pydata.org/pandas-docs... https://pandas.pydata.org/pandas-docs... https://pandas.pydata.org/pandas-docs... * 실습코드 : https://github.com/corazzon/cracking-... * https://pandas.pydata.org/Pandas_Chea... * 판다스 10분 완성 : https://dataitgirls2.github.io/10minu...

* 실습코드 : https://github.com/corazzon/cracking-... Pandas 기초 - 파이썬 판다스로 pd.concat([df1,df2]) 시리즈, 데이터프레임 합치기 * https://pandas.pydata.org/Pandas_Chea... * 판다스 10분 완성 : https://dataitgirls2.github.io/10minu...

* 실습코드 : https://github.com/corazzon/cracking-... 판다스는 파이썬에서 사용할 수 있는 엑셀과 비슷한 툴입니다. melt, pivot 으로 Tidy Data 만들기와 메소드 체이닝(Method Chaining)을 통해 sort_values, rename, sort_index, reset_index로 데이터 프레임 Reshaping 실습을 해봅니다. * https://pandas.pydata.org/Pandas_Chea... * 판다스 10분 완성 : https://dataitgirls2.github.io/10minu...

Pandas 기초 - 파이썬 판다스 df.sort_values, rename, sort_index, reset_index로 데이터 프레임 Reshaping 튜토리얼 강의 * 실습코드 : https://github.com/corazzon/cracking-... df.sort_values, rename, sort_index, reset_index 로 데이터 프레임 Reshaping * https://pandas.pydata.org/Pandas_Chea... * 판다스 10분 완성 : https://dataitgirls2.github.io/10minu...

Pandas 기초 - 파이썬 판다스 assign 으로 새로운 컬럼 만들기, qcut으로 binning, bucketing 하기 튜토리얼 강의 * 실습코드 : https://github.com/corazzon/cracking-... df.assign(Area=lambda df: df.Length*df.Height) df['Volume'] = df.Length*df.Height*df.Depth pd.qcut(df.col, n, labels=False) * https://pandas.pydata.org/Pandas_Chea... * 판다스 10분 완성 : https://dataitgirls2.github.io/10minu...

Pandas 기초 - Pandas Handling Missing Data 튜토리얼 강의 * 실습코드 : https://github.com/corazzon/cracking-... * https://pandas.pydata.org/Pandas_Chea... * 판다스 10분 완성 : https://dataitgirls2.github.io/10minu...

Pandas 기초 - 파이썬 판다스로 apply 활용하기 lambda 익명함수 사용하기 튜토리얼 강의 * 실습코드 : https://github.com/corazzon/cracking-... * https://pandas.pydata.org/Pandas_Chea... * 판다스 10분 완성 : https://dataitgirls2.github.io/10minu...

Pandas 기초 - 파이썬 판다스로 기본 통계 하기 value_counts, nunique, sum, count, mean, median 튜토리얼 강의 * 실습코드 : https://github.com/corazzon/cracking-... * https://pandas.pydata.org/Pandas_Chea... * 판다스 10분 완성 : https://dataitgirls2.github.io/10minu...

Pandas 기초 - 일부 컬럼을 기준으로 데이터 가져오기 Subset Variables (Columns) 튜토리얼 강의 * 실습코드 : https://github.com/corazzon/cracking-... * https://pandas.pydata.org/Pandas_Chea... * 판다스 10분 완성 : https://dataitgirls2.github.io/10minu...

Pandas 기초 - head, tail로 데이터 미리보기 df.sample(frac=0.5), df.sample(n=10), df.nlargest, df.nsmallest 튜토리얼 강의 * 실습코드 : https://github.com/corazzon/cracking-... * https://pandas.pydata.org/Pandas_Chea... * 판다스 10분 완성 : https://dataitgirls2.github.io/10minu... * 특정 비율과 갯수로 데이터를 샘플링 하기 df.sample(frac=0.5) Randomly select fraction of rows. df.sample(n=10) Randomly select n rows. * 인덱스의 순서로 데이터를 색인해 오기 df.iloc[ 10:20 ] Select rows by position. * 특정 컬럼에서 가장 큰 값과 작은 값 가져오기 df.nlargest(n, 'value') Select and order top n entries. df.nsmallest(n, 'value') Select and order bottom n entries.

[4] Pandas 기초 - and, or, not, xor, any, all 연산 이해하기 튜토리얼 강의 * 실습코드 : https://github.com/corazzon/cracking-... * 판다스 10분 완성 : https://dataitgirls2.github.io/10minu... * https://pandas.pydata.org/Pandas_Chea...

판다스 데이터프레임 비교연산자로 색인하기, drop_duplicates() * 실습코드 : https://github.com/corazzon/cracking-... * https://pandas.pydata.org/Pandas_Chea... * 판다스 10분 완성 : https://dataitgirls2.github.io/10minu...

- 판다스 10분 완성소개, cheat sheet 보고 데이터프레임 만들어보기 * 실습코드 : https://github.com/corazzon/cracking-... * 판다스 10분 완성 : https://dataitgirls2.github.io/10minu... * https://pandas.pydata.org/Pandas_Chea...

- 판다스 10분 완성소개, cheat sheet 보고 데이터프레임 만들어보기 * 실습코드 : https://github.com/corazzon/cracking-... * https://pandas.pydata.org/Pandas_Chea... * 판다스 10분 완성 : https://dataitgirls2.github.io/10minu...

PyData DC 2018 The world wide web contains text in many languages and modern systems often cannot be restricted to a single locale. Being able to make use of the text in other languages requires a pipeline that can scale. We'll describe and demonstrate how we can create a streaming pipeline to consume, preprocess, and translate the streaming text. === www.pydata.org PyData is an educational program of NumFOCUS, a 501(c)3 non-profit organization in the United States. PyData provides a forum for the international community of users and developers of data analysis tools to share ideas and learn from each other. The global PyData network promotes discussion of best practices, new approaches, and emerging technologies for data management, processing, analytics, and visualization. PyData communities approach data science using many languages, including (but not limited to) Python, Julia, and R. PyData conferences aim to be accessible and community-driven, with novice to advanc...

PyData DC 2018 We will be using Beautiful Soup to Webscrape the IMDB website and create a function that will allow you to create a dictionary object on specific metadata of the IMDB profile for any IMDB ID you pass through as an argument. Slides: https://www.slideshare.net/PyData/pyd... === www.pydata.org PyData is an educational program of NumFOCUS, a 501(c)3 non-profit organization in the United States. PyData provides a forum for the international community of users and developers of data analysis tools to share ideas and learn from each other. The global PyData network promotes discussion of best practices, new approaches, and emerging technologies for data management, processing, analytics, and visualization. PyData communities approach data science using many languages, including (but not limited to) Python, Julia, and R. PyData conferences aim to be accessible and community-driven, with novice to advanced level presentations. PyData tutorials and talks bring attendee...

PyData Ann Arbor Meetup - May 13, 2019 Sponsored by NumFOCUS, TD Ameritrade, and MIDAS https://www.meetup.com/PyData-Ann-Arbor/ Before AI, before machine learning and pipelines, and before dashboards and BI, an organization starts with a pile of data, some business questions, and a few ideas on how to connect the two -- a greenfield, and an entry point for data science. Answering business questions and turning raw data into insights, models, and products means more than just writing code and doing analysis. A successful data science team needs tools, a communication strategy, thoughtful infrastructure, and a plan to deliver on their goals. This talk will cover how to tackle greenfield data science challenges from the perspective of the first data science hire in an organization, and how to build data science infrastructure from the ground up. —————————————————— Caitlin Hudon is Lead Data Scientist at OnlineMedEd, a startup in the Edtech space in Austin. Her 8+ years of appli...

PyData DC 2018 Leaflet.js, a JavaScript visualization library, makes maps that are as lightweight and flexible as they are interactive and web-friendly. Folium brings that capability to Python, so we can use our favorite tools to ingest, join and process location data from the Census, WMATA, and Walkscore APIs and end up with HTML/JS maps that can be rendered in a notebook or served programmatically via web app. Slides - https://GitHub.com/gotoariel/folium-demo === www.pydata.org PyData is an educational program of NumFOCUS, a 501(c)3 non-profit organization in the United States. PyData provides a forum for the international community of users and developers of data analysis tools to share ideas and learn from each other. The global PyData network promotes discussion of best practices, new approaches, and emerging technologies for data management, processing, analytics, and visualization. PyData communities approach data science using many languages, including (but not limite...

PyData NYC 2018 Conda-forge is a community-driven, cross-platform effort to package the libraries we all use. Learn how conda-forge works and how you can use it as a resource to package and distribute your code and dependencies! === www.pydata.org PyData is an educational program of NumFOCUS, a 501(c)3 non-profit organization in the United States. PyData provides a forum for the international community of users and developers of data analysis tools to share ideas and learn from each other. The global PyData network promotes discussion of best practices, new approaches, and emerging technologies for data management, processing, analytics, and visualization. PyData communities approach data science using many languages, including (but not limited to) Python, Julia, and R. PyData conferences aim to be accessible and community-driven, with novice to advanced level presentations. PyData tutorials and talks bring attendees the latest project features along with cutting-edge use ca...

PyData NYC 2018 This is an overview of what I learned about web apps while building the Trending service for ShopRunner. It will focus on details that are critical to performant, production systems but that are often left out of the online literature aimed at data scientists. This will include dealing with updating data, enabling scrolling feeds, errors and the canonical use of HTTP status codes, and performance. === www.pydata.org PyData is an educational program of NumFOCUS, a 501(c)3 non-profit organization in the United States. PyData provides a forum for the international community of users and developers of data analysis tools to share ideas and learn from each other. The global PyData network promotes discussion of best practices, new approaches, and emerging technologies for data management, processing, analytics, and visualization. PyData communities approach data science using many languages, including (but not limited to) Python, Julia, and R. PyData conferences ai...

PyData NYC 2018 At Monetate, we’ve deployed Bayesian bandits (both noncontextual and contextual) to help our clients optimize their e-commerce sites since early 2016. This talk is an overview of the lessons we’ve learned from both the processes of deploying real-time Bayesian machine learning systems at scale and building a data product on top of these systems that is accessible to non-technical users (marketers) === www.pydata.org PyData is an educational program of NumFOCUS, a 501(c)3 non-profit organization in the United States. PyData provides a forum for the international community of users and developers of data analysis tools to share ideas and learn from each other. The global PyData network promotes discussion of best practices, new approaches, and emerging technologies for data management, processing, analytics, and visualization. PyData communities approach data science using many languages, including (but not limited to) Python, Julia, and R. PyData conferences ai...

PyData NYC 2018 This talk focuses on an extremely experimental library called slumba that attemps to bring the convenience and speed of numba to user-defined scalar, aggregate and window functions in a SQLite database. === www.pydata.org PyData is an educational program of NumFOCUS, a 501(c)3 non-profit organization in the United States. PyData provides a forum for the international community of users and developers of data analysis tools to share ideas and learn from each other. The global PyData network promotes discussion of best practices, new approaches, and emerging technologies for data management, processing, analytics, and visualization. PyData communities approach data science using many languages, including (but not limited to) Python, Julia, and R. PyData conferences aim to be accessible and community-driven, with novice to advanced level presentations. PyData tutorials and talks bring attendees the latest project features along with cutting-edge use cases.

PyData London Meetup #52 Tuesday, January 8, 2018 Sponsored & Hosted by Man AHL **** www.pydata.org PyData is an educational program of NumFOCUS, a 501(c)3 non-profit organization in the United States. PyData provides a forum for the international community of users and developers of data analysis tools to share ideas and learn from each other. The global PyData network promotes discussion of best practices, new approaches, and emerging technologies for data management, processing, analytics, and visualization. PyData communities approach data science using many languages, including (but not limited to) Python, Julia, and R. PyData conferences aim to be accessible and community-driven, with novice to advanced level presentations. PyData tutorials and talks bring attendees the latest project features along with cutting-edge use cases.

PyData NYC 2018 Train, evaluate, and deploy deep learning models with cloud compute through API calls. === www.pydata.org PyData is an educational program of NumFOCUS, a 501(c)3 non-profit organization in the United States. PyData provides a forum for the international community of users and developers of data analysis tools to share ideas and learn from each other. The global PyData network promotes discussion of best practices, new approaches, and emerging technologies for data management, processing, analytics, and visualization. PyData communities approach data science using many languages, including (but not limited to) Python, Julia, and R. PyData conferences aim to be accessible and community-driven, with novice to advanced level presentations. PyData tutorials and talks bring attendees the latest project features along with cutting-edge use cases.

PyData NYC 2018 TileDB is an open-source storage manager for multi-dimensional sparse and dense array data. It has a novel architecture that addresses some of the pain points in storing array data on “big-data” and “cloud” storage architectures. This talk will highlight TileDB’s design and its ability to integrate with analysis environments relevant to the PyData community such as Python, R, Julia, etc. Slides - https://www.slideshare.net/PyData/the... === www.pydata.org PyData is an educational program of NumFOCUS, a 501(c)3 non-profit organization in the United States. PyData provides a forum for the international community of users and developers of data analysis tools to share ideas and learn from each other. The global PyData network promotes discussion of best practices, new approaches, and emerging technologies for data management, processing, analytics, and visualization. PyData communities approach data science using many languages, including (but not limited to) Pyt...

PyData NYC 2018 A comprehensive tutorial on CatBoost ( http://catboost.ai ), a new open-source gradient boosting library, that outperforms existing publicly available implementations of gradient boosting in terms of quality, has the fastest applier algorithm and the fastest GPU training of all publicly available GBDT implementations. === www.pydata.org PyData is an educational program of NumFOCUS, a 501(c)3 non-profit organization in the United States. PyData provides a forum for the international community of users and developers of data analysis tools to share ideas and learn from each other. The global PyData network promotes discussion of best practices, new approaches, and emerging technologies for data management, processing, analytics, and visualization. PyData communities approach data science using many languages, including (but not limited to) Python, Julia, and R. PyData conferences aim to be accessible and community-driven, with novice to advanced level presentati...

PyData NYC 2018 Machine learning models are increasingly used to make decisions that affect people’s lives. With this power comes a responsibility to ensure that model predictions are fair. In this talk I’ll introduce several common model fairness metrics, discuss their tradeoffs, and finally demonstrate their use with a case study analyzing anonymized data from one of Civis Analytics’s client engagements. Slides - http://hinnefe2.github.io/talks/PyDat... === www.pydata.org PyData is an educational program of NumFOCUS, a 501(c)3 non-profit organization in the United States. PyData provides a forum for the international community of users and developers of data analysis tools to share ideas and learn from each other. The global PyData network promotes discussion of best practices, new approaches, and emerging technologies for data management, processing, analytics, and visualization. PyData communities approach data science using many languages, including (but not limited to) P...

PyData NYC 2018 Intake is a simple library providing a single interface for cataloging, describing and reading any kind of data. Catalogs give end-users an easy way to find data, locally, in a cloud service, or on an Intake server. Thus, Intake separates the definition of data sources from their use and analysis, so that Data Engineers and Data Scientists can get on with their respective jobs. Slides - https://docs.google.com/presentation/... === www.pydata.org PyData is an educational program of NumFOCUS, a 501(c)3 non-profit organization in the United States. PyData provides a forum for the international community of users and developers of data analysis tools to share ideas and learn from each other. The global PyData network promotes discussion of best practices, new approaches, and emerging technologies for data management, processing, analytics, and visualization. PyData communities approach data science using many languages, including (but not limited to) Python, Julia...