#progresstoday — Public Fediverse posts

Rearranged my pinned mastodon hashtags and feeling more focused 😎

#ProgressToday

Learning from #EffectivePandas and #PythonForDataAnalysis.

Recipe for permutating or randomly reordering the rows of a DataFrame or Series:
new_order = np.random.permutation(n)
df.iloc[new_order]
df.take(new_order)
To permutate the cols of a DataFrame, add "axis='columns'" to .take().

Method for selecting a random subset of the rows DataFrame or Series:
df.sample(n=, frac=)
To allow for replacement, add "replace=True" to .sample().

#LearnPython #ProgressToday

Learning from #EffectivePandas and #PythonForDataAnalysis.

Recipe for permutating or randomly reordering the rows of a DataFrame or Series:
new_order = np.random.permutation(n)
df.iloc[new_order]
df.take(new_order)
To permutate the cols of a DataFrame, add "axis='columns'" to .take().

Method for selecting a random subset of the rows DataFrame or Series:
df.sample(n=, frac=)
To allow for replacement, add "replace=True" to .sample().

#LearnPython #ProgressToday

Learning from #EffectivePandas and #PythonForDataAnalysis.

The preferred way to index and filter a Series or a DataFrame is i) with .loc[] indexing on index labels or ii) with .iloc[] indexing on index position integers. Their call signatures are nearly identical:

.loc[rows]
.loc[:, cols]
.loc[rows, cols]

Their strengths come from the increased clarity what we intend to index on and what we intend to select, therefore helping us not be the problem 😂

#LearnPython #ProgressToday

#ProgressToday Learning from #EffectivePandas and #PythonForDataAnalysis.

The preferred way to index and filter a Series or a DataFrame is i) with .loc[] indexing on index labels or ii) with .iloc[] indexing on index position integers. Their call signatures are nearly identical:

.loc[rows]
.loc[:, cols]
.loc[rows, cols]

Their strengths come from the increased clarity what we intend to index on and what we intend to select, therefore helping us not be the problem 😂

#LearnPython

Continued my way through #EffectivePandas and #PythonForDataAnalysis.

Element-wise transformation of a Series values or an Index labels can be done by feeding a dictionary (for selected elements) or a function (for all elements) into method

.map(dict or func)

Binning of a Series or column can be done with i) the data values, or ii) the data quantiles:

.cut(data, bins or nbins, right=, labels=, precision=)
.qcut(data, quantiles or nquartiles)

#LearnPython #ProgressToday

#ProgressToday Here are two methods that change the values of a Series or column:

.replace(to_replace=, value=, regex=)
.clip(lower=, upper=)

The former is more general, while the latter is for numerical data types to clip outliers or extreme values.

Disambiguation: when a Series or column has string data, .replace() changes whole strings, whereas .str.replace() changes sub-strings.

#LearnPython

#ProgressToday Finished the sections in #EffectivePandas and #PythonForDataAnalysis on converting the data types of a Series or column. Top methods:

.astype(dtype, copy=, errors=)
.convert_dtypes()
pd.to_datetime()
pd.CategoricalDtype(categories=, ordered=)

The 1st one converts to Python + NumPy types, while the 2nd one converts to pandas extension types that support NA.

Before converting data types, be sure to take care of codes for missing data or errors.

#LearnPython

#ProgressToday Finished going through sections in #EffectivePandas and #PythonForDataAnalysis related to duplicated data and cleaning. It's good that the two important methods apply to all three objects - Series, DataFrame, and Index:

.duplicated(subset=, keep=)
.drop_duplicates(subset=, keep=)

One difference is that the kwarg 'subset=' applies to DataFrame objects only, which can have multiple columns to choose from.

#LearnPython

#ProgressToday Finished going over sections in #EffectivePandas and #PythonForDataAnalysis related to handling missing data. Here are useful methods on this topic:

.isna()
.notna()
.dropna(how=, thresh=, axis=)
.fillna(value=, method=, limit=, axis=)
.interpolate(method=, limit=, axis=)

#LearnPython

#ProgressToday Spent some time tidying up my recent code. Cleaned up some code snippets from my latest notebooks and put them in separate #Python files so that they can be run in one go. Also added one-liner docstrings to help my future self understand.

These habits should help boost my productivity over time. We'll see!

#ProgressToday: Finished reading chapter 3 "Pythonic Syntax and Common Pitfalls" in #MasteringPython. The common pitfalls didn't surprise me as much now. Learned of the "walrus operator" and "switch statement". Installed #pycodestyle, tried it on a couple of my python files, and fixed the issues to get a clean pass.

Going forward, I'll do these:

I. Use pycodestyle to scan my python files as I work on them;

II. Look for opportunities to use "walrus operator" and "switch statement".

#Python

#ProgressToday: Finished reading chapter 10 "Testing and Logging" in #MasteringPython. Used #Python doctest and logging modules a little as in the examples. Decided to take the minimalist approach here:

I. Start using logging module with basicConfig in my scripting;

II. Look for opportunities to practice doctest also in my scripting.

#ProgressToday: Finished reading chapter 9 "Documentation" in #MasteringPython. Used Sphinx with a simplest project and gained a little experience of current, real-world documentation. Decided I'll go a minimalist approach:

I. For my external-facing functions or classes , I'll write at least a one-liner docstring to start with.

II. For my Jupyter notebooks, in addition, I'll use markdowns to comment on requirements, input data, data cleaning, plots, etc.

#Python

#ProgressToday: Installed Python 3.11, created a venv, and installed Jupyter in it. Keep going!

#ProgressToday: Finished working through the 25 recipes of chapter 8 "Classes and Objects" in #PythonCookbook. While some concepts were hard to appreciate at first, e.g. descriptors, revisiting them for a second time made the difference. Also, chapter 7 "Classes and OOP" in #PythonDistilled helped me grasp the concepts from a higher level. It's good to learn different ways of defining classes and advanced ways of customizing and optimizing them. I look forward to using these recipes! #Python

I did a quick #introduction on 11/6, and feel the need to do a slow one.

My wife and I are raising a 4YO and I toot #parenting, #ParentingJoy, #Preschooler to share with other parents.

I'm a practicing pythonista (#Python) and working my way through #PythonCookbook and #PythonDistilled. I toot #ProgressToday to record my progress. Right, maybe you didn't know - You are my accountability partners. Thanks!

I enjoy good humor. Sometimes my reply doesn't make sense, hopefully it makes you smile.

I did a quick #introduction on 11/6, and feel the need to do a slow one.

My wife and I are raising a 4YO and I toot #parenting, #ParentingJoy, #Preschooler to share with other parents.

I'm a practicing pythonista (#Python) and working my way through #PythonCookbook and #PythonDistilled. I toot #ProgressToday to record my progress. Right, maybe you didn't know - You are my accountability partners. Thanks!

I enjoy good humor. Sometimes my reply doesn't make sense, hopefully it makes you smile.

#ProgressToday: Finished reading chapter 7 "Classes and OOP" in #PythonDistilled and worked through ~1/3 of the recipes of chapter 8 "Classes and Objects" in #PythonCookbook. Feeling updated in the relevant concepts and patterns in #Python. While more advanced ideas are still brain bending, simpler ones are starting to make sense.

BTW, the above was yesterday's summary, which seems to have been lost during the @sfba.social server team's attempted upgrade to Mastodon v4 last night.

#ProgressToday: Finished reading chapter 7 "Classes and OOP" in #PythonDistilled and worked through ~1/3 of the recipes in chapter 8 "Classes and Objects" in #PythonCookbook. Feeling updated in a number of #Python concepts and patterns with respect to classes and objects. While some more advanced ones are still brain bending, some simpler ones are already making sense. Good to make progress! 😎

#ProgressToday: Paused reading the big chapter in #PythonDistilled, and started working through the examples in the corresponding chapter in #PythonCookbook. Ran into a problem with one example, and was delighted to find relevant explanations in the other book and complete it. One book is good, two books are great! 😎

#Python

#ProgressToday: Reached 2/3 of the big chapter in #PythonDistilled and encountered a number of new concepts. It's good to pause and digest for now.

#Python

#ProgressToday: Reading close to the mid point of a big chapter in #PythonDistilled, and feeling clarity and conciseness is indeed valuable. Look forward to reading the rest of this chapter.

#Python

#ProgressToday: Fiddled the advanced mastodon UI and figured out how to pin a hashtag.

Feeling good about it and thinking it may be a useful hint to the group of folks in #twittermigration .

#ProgressToday: Switched to advanced UI in mastodon and got a bit lost again. No problem, I'll figure it out tomorrow. 😎