Data Handling using Pandas - I

Overview

Learning outcomes

• Students will be able to explain why Pandas is used alongside NumPy and Matplotlib, and import it with the alias pd.
• Students will be able to create a Series from a list, an ndarray, a dictionary and a scalar value, with or without a custom index.
• Students will be able to use Series attributes and methods (index, values, dtype, shape, size, name, empty, head(), tail(), count()).
• Students will be able to perform mathematical operations on Series and predict where index alignment produces NaN.
• Students will be able to select, index and slice Series and DataFrames using labels (loc), positions (iloc) and Boolean conditions.
• Students will be able to create DataFrames from a dictionary of lists, a dictionary of Series, a list of dictionaries and a CSV file.
• Students will be able to add, select, delete and rename rows and columns, and iterate over a DataFrame with iterrows() and items().
• Students will be able to import and export data with read_csv() and to_csv() using sep, header, index, names and index_col.

Lesson plan

Notes

import pandas as pd
import numpy as np
arr = np.array([72, 85, 64])
print(arr)
print(pd.Series(arr, index=['Ravi', 'Sana', 'Tara']))
print(pd.DataFrame({'Marks': arr, 'House': ['Red', 'Blue', 'Red']}))

[72 85 64]
Ravi    72
Sana    85
Tara    64
dtype: int64
   Marks House
0     72   Red
1     85  Blue
2     64   Red

import pandas as pd
runs = pd.Series([45, 102, 7, 63])
print(runs)
print()
runs = pd.Series([45, 102, 7, 63], index=['Rohit', 'Virat', 'Shubman', 'Rishabh'])
print(runs)
print()
fee = pd.Series(1500, index=['Jan', 'Feb', 'Mar'])
print(fee)

0     45
1    102
2      7
3     63
dtype: int64

Rohit       45
Virat      102
Shubman      7
Rishabh     63
dtype: int64

Jan    1500
Feb    1500
Mar    1500
dtype: int64

import pandas as pd
import numpy as np
temps = np.array([38.5, 41.2, 43.0, 39.8])
s = pd.Series(temps, index=['Jaipur', 'Nagpur', 'Banda', 'Delhi'])
print(s)
print()
try:
    pd.Series(np.array([1, 2, 3]), index=['x', 'y'])
except ValueError as e:
    print('ValueError:', e)

Jaipur    38.5
Nagpur    41.2
Banda     43.0
Delhi     39.8
dtype: float64

ValueError: Length of values (3) does not match length of index (2)

import pandas as pd
capitals = {'Kerala': 'Thiruvananthapuram', 'Punjab': 'Chandigarh', 'Assam': 'Dispur'}
s = pd.Series(capitals)
print(s)
print()
medals = {'India': 6, 'Kenya': 11, 'Japan': 45}
m = pd.Series(medals, index=['Japan', 'India', 'Nepal'])
print(m)

Kerala    Thiruvananthapuram
Punjab            Chandigarh
Assam                 Dispur
dtype: object

Japan    45.0
India     6.0
Nepal     NaN
dtype: float64

import pandas as pd
s = pd.Series([41, 39, 44, 40], index=['Mon', 'Tue', 'Wed', 'Thu'])
s.name = 'MaxTemp'
s.index.name = 'Day'
print(s)
print(s.index)
print(s.values)
print(s.dtype, s.shape, s.size, s.ndim)
print(s.empty, pd.Series(dtype='float64').empty)

Day
Mon    41
Tue    39
Wed    44
Thu    40
Name: MaxTemp, dtype: int64
Index(['Mon', 'Tue', 'Wed', 'Thu'], dtype='object', name='Day')
[41 39 44 40]
int64 (4,) 4 1
False True

import pandas as pd
rain = pd.Series([722, 2955, 603, 1092], index=['Pune', 'Cherrapunji', 'Agra', 'Bhopal'])
print(rain['Agra'])
print(rain.loc['Pune'])
print(rain.iloc[1])
print(rain.iloc[-1])
print(rain.loc[['Agra', 'Pune']])
print(rain.iloc[[0, 3]])

603
722
2955
1092
Agra    603
Pune    722
dtype: int64
Pune       722
Bhopal    1092
dtype: int64

import pandas as pd
s = pd.Series([10, 20, 30, 40, 50], index=['a', 'b', 'c', 'd', 'e'])
print(s[1:3])
print(s['b':'d'])
print(s.iloc[::-2])
s['c':'d'] = 0
print(s.tolist())

b    20
c    30
dtype: int64
b    20
c    30
d    40
dtype: int64
e    50
c    30
a    10
dtype: int64
[10, 20, 0, 0, 50]

import pandas as pd
temp = pd.Series([38, 44, 41, 36, 45], index=['Mon', 'Tue', 'Wed', 'Thu', 'Fri'])
print(temp > 40)
print(temp[temp > 40])
print(temp[(temp > 37) & (temp < 42)])
print(temp[~(temp > 40)].index.tolist())

Mon    False
Tue     True
Wed     True
Thu    False
Fri     True
dtype: bool
Tue    44
Wed    41
Fri    45
dtype: int64
Mon    38
Wed    41
dtype: int64
['Mon', 'Thu']

import pandas as pd
import numpy as np
s = pd.Series([12, 25, np.nan, 31, 47, 58, 66])
print(s.head())
print(s.tail(2))
print(s.size, s.count())

0    12.0
1    25.0
2     NaN
3    31.0
4    47.0
dtype: float64
5    58.0
6    66.0
dtype: float64
7 6

import pandas as pd
shopA = pd.Series([40, 25, 60], index=['pen', 'pencil', 'notebook'])
shopB = pd.Series([35, 50, 20], index=['notebook', 'eraser', 'pen'])
print(shopA * 2)
print(shopA + shopB)
print(shopA.add(shopB, fill_value=0))
print(shopA.sub(shopB, fill_value=0)['eraser'])

pen          80
pencil       50
notebook    120
dtype: int64
eraser       NaN
notebook    95.0
pen         60.0
pencil       NaN
dtype: float64
eraser      50.0
notebook    95.0
pen         60.0
pencil      25.0
dtype: float64
-50.0

import pandas as pd
data = {'Name': ['Aarav', 'Diya', 'Kabir'],
        'Maths': [78, 91, 65],
        'Science': [82, 88, 70]}
df = pd.DataFrame(data, index=['S1', 'S2', 'S3'])
print(df)
print()
rows = [[12951, 'Rajdhani', 'Mumbai'], [12301, 'Rajdhani', 'Howrah'], [22439, 'Vande Bharat', 'Katra']]
trains = pd.DataFrame(rows, columns=['TrainNo', 'Type', 'To'])
print(trains)
print(pd.DataFrame().empty)

     Name  Maths  Science
S1  Aarav     78       82
S2   Diya     91       88
S3  Kabir     65       70

   TrainNo          Type      To
0    12951      Rajdhani  Mumbai
1    12301      Rajdhani  Howrah
2    22439  Vande Bharat   Katra
True

import pandas as pd
records = [{'Name': 'Ananya', 'House': 'Tagore', 'Points': 18},
           {'Name': 'Vihaan', 'House': 'Raman'},
           {'Name': 'Sara', 'House': 'Tagore', 'Points': 25}]
df = pd.DataFrame(records)
print(df)
print(df.dtypes)

     Name   House  Points
0  Ananya  Tagore    18.0
1  Vihaan   Raman     NaN
2    Sara  Tagore    25.0
Name       object
House      object
Points    float64
dtype: object

import pandas as pd
maths = pd.Series([78, 91, 65], index=['Aarav', 'Diya', 'Kabir'])
hindi = pd.Series([88, 72, 69], index=['Diya', 'Kabir', 'Meera'])
df = pd.DataFrame({'Maths': maths, 'Hindi': hindi})
print(df)

       Maths  Hindi
Aarav   78.0    NaN
Diya    91.0   88.0
Kabir   65.0   72.0
Meera    NaN   69.0

import pandas as pd
df = pd.read_csv('marks.csv')
print(df.head(3))
print()
print(df.tail(2))
print()
print(df.head(-3))

   RollNo   Name  Maths  Science  English
0       1  Aarav     78       82       74
1       2   Diya     91       88       95
2       3  Kabir     65       70       60

   RollNo   Name  Maths  Science  English
3       4  Meera     88       79       85
4       5  Rohan     55       62       58

   RollNo   Name  Maths  Science  English
0       1  Aarav     78       82       74
1       2   Diya     91       88       95

import pandas as pd
df = pd.DataFrame({'Name': ['Ishaan', 'Tanvi', 'Farhan'],
                   'Section': ['A', 'B', 'A'],
                   'Percent': [86.4, 92.0, 74.5]},
                  index=['A101', 'A102', 'A103'])
print(df.index)
print(df.columns)
print(df.dtypes)
print(df.shape, df.size, df.ndim, df.empty)
print(df.values)
print(df.T)

Index(['A101', 'A102', 'A103'], dtype='object')
Index(['Name', 'Section', 'Percent'], dtype='object')
Name        object
Section     object
Percent    float64
dtype: object
(3, 3) 9 2 False
[['Ishaan' 'A' 86.4]
 ['Tanvi' 'B' 92.0]
 ['Farhan' 'A' 74.5]]
           A101   A102    A103
Name     Ishaan  Tanvi  Farhan
Section       A      B       A
Percent    86.4   92.0    74.5

import pandas as pd
df = pd.DataFrame({'Name': ['Aarav', 'Diya', 'Kabir'],
                   'Maths': [78, 91, 65],
                   'Science': [82, 88, 70]}, index=['S1', 'S2', 'S3'])
print(df['Maths'])
print(df[['Name', 'Science']])
print(df.loc['S2'])
print(df.loc['S2', 'Maths'], df.iloc[2, 2])

S1    78
S2    91
S3    65
Name: Maths, dtype: int64
     Name  Science
S1  Aarav       82
S2   Diya       88
S3  Kabir       70
Name       Diya
Maths        91
Science      88
Name: S2, dtype: object
91 70

import pandas as pd
df = pd.read_csv('marks.csv', index_col='RollNo')
print(df.loc[2:4, 'Maths':'English'])
print()
print(df.iloc[1:3, 0:2])
print()
print(df[0:2])

        Maths  Science  English
RollNo                         
2          91       88       95
3          65       70       60
4          88       79       85

         Name  Maths
RollNo              
2        Diya     91
3       Kabir     65

         Name  Maths  Science  English
RollNo                                
1       Aarav     78       82       74
2        Diya     91       88       95

import pandas as pd
df = pd.read_csv('marks.csv', index_col='RollNo')
print(df[df['Maths'] >= 75])
print()
print(df.loc[(df.Science > 80) | (df.English < 60), ['Name', 'Science', 'English']])
print()
entries = pd.DataFrame({'Name': ['Riya', 'Arjun', 'Riya', 'Neha'],
                        'Event': ['100m', 'Chess', '100m', 'Chess']})
print(entries[entries.duplicated()])
print(entries[~entries.duplicated(subset=['Event'])])

         Name  Maths  Science  English
RollNo                                
1       Aarav     78       82       74
2        Diya     91       88       95
4       Meera     88       79       85

         Name  Science  English
RollNo                         
1       Aarav       82       74
2        Diya       88       95
5       Rohan       62       58

   Name Event
2  Riya  100m
    Name  Event
0   Riya   100m
1  Arjun  Chess

import pandas as pd
df = pd.DataFrame({'Name': ['Aarav', 'Diya'], 'Maths': [78, 91], 'Science': [82, 88]})
df['Total'] = df['Maths'] + df['Science']
df['Class'] = 'XII-A'
df.loc[len(df)] = ['Kabir', 65, 70, 135, 'XII-A']
df.insert(1, 'Roll', [1, 2, 3])
print(df)

    Name  Roll  Maths  Science  Total  Class
0  Aarav     1     78       82    160  XII-A
1   Diya     2     91       88    179  XII-A
2  Kabir     3     65       70    135  XII-A

import pandas as pd
df = pd.read_csv('marks.csv', index_col='RollNo')
print(df.drop(3))
print()
print(df.drop(['Science', 'English'], axis=1).columns.tolist())
df = df.drop(index=[1, 2])
del df['English']
print(df)

         Name  Maths  Science  English
RollNo                                
1       Aarav     78       82       74
2        Diya     91       88       95
4       Meera     88       79       85
5       Rohan     55       62       58

['Name', 'Maths']
         Name  Maths  Science
RollNo                       
3       Kabir     65       70
4       Meera     88       79
5       Rohan     55       62

import pandas as pd
df = pd.DataFrame({'nm': ['Gill', 'Jadeja'], 'r': [91, 44], 'b': [102, 38]})
df = df.rename(columns={'nm': 'Player', 'r': 'Runs', 'b': 'Balls'})
df = df.rename(index={0: 'P1', 1: 'P2'})
print(df)
df.columns = ['Batter', 'R', 'B']
print(df.columns.tolist())

    Player  Runs  Balls
P1    Gill    91    102
P2  Jadeja    44     38
['Batter', 'R', 'B']

import pandas as pd
df = pd.DataFrame({'Name': ['Aarav', 'Diya'], 'Maths': [78, 91], 'Science': [82, 88]})
for label, row in df.iterrows():
    print(label, row['Name'], row['Maths'] + row['Science'])
for col, values in df.items():
    print(col, values.tolist())

0 Aarav 160
1 Diya 179
Name ['Aarav', 'Diya']
Maths [78, 91]
Science [82, 88]

import pandas as pd
df = pd.read_csv('marks.csv', index_col='RollNo', nrows=3)
print(df)
print()
st = pd.read_csv('stations.txt', sep=';', header=None,
                 names=['Code', 'Station', 'State', 'Platforms'], index_col='Code')
print(st)

         Name  Maths  Science  English
RollNo                                
1       Aarav     78       82       74
2        Diya     91       88       95
3       Kabir     65       70       60

              Station        State  Platforms
Code                                         
NDLS        New Delhi        Delhi         16
BCT    Mumbai Central  Maharashtra          9
MAS   Chennai Central   Tamil Nadu         17
HWH            Howrah  West Bengal         23

import pandas as pd
df = pd.read_csv('result.csv')
top = df[df['Percent'] >= 85]
top.to_csv('toppers.csv', index=False)
print(open('toppers.csv').read())
print(top.to_csv(sep='|', header=False))

Adm,Name,Class,Percent
A101,Ishaan,12A,86.4
A102,Tanvi,12B,92.0
A106,Pooja,12A,95.6

0|A101|Ishaan|12A|86.4
1|A102|Tanvi|12B|92.0
5|A106|Pooja|12A|95.6

import pandas as pd
secA = pd.DataFrame({'Name': ['Ira', 'Dev'], 'Marks': [81, 77]})
secB = pd.DataFrame({'Name': ['Zoya'], 'Marks': [90], 'Club': ['Robotics']})
print(pd.concat([secA, secB], ignore_index=True))

   Name  Marks      Club
0   Ira     81       NaN
1   Dev     77       NaN
2  Zoya     90  Robotics

Key terms

Library

A collection of ready-made modules that can be imported into a program, such as NumPy, Pandas and Matplotlib.

Pandas

A Python library for analysing labelled, tabular data; its name comes from 'panel data'.

Series

A one-dimensional labelled array in which all values share one data type.

DataFrame

A two-dimensional labelled table of rows and columns; each column is a Series and columns may have different data types.

Index

The labels attached to the rows of a Series or DataFrame.

Column label

The name of a DataFrame column, stored in df.columns.

ndarray

NumPy's N-dimensional array of values of a single type; it can be used to create a Series or DataFrame.

NaN

'Not a Number'; the floating-point marker Pandas uses for a missing value.

dtype

The data type of a Series or column, e.g. int64, float64, object.

Attribute

A property of an object read without brackets, e.g. df.shape.

Alignment

Matching elements by index label (not by position) when two Series or DataFrames are combined.

loc

Accessor that selects rows and columns by label; label slices include the end.

iloc

Accessor that selects rows and columns by integer position; slices exclude the end.

Slicing

Selecting a continuous range of elements or rows using start:stop:step.

Boolean indexing

Selecting the elements or rows where a True/False condition is True.

Transpose

Swapping rows and columns of a DataFrame, obtained with df.T.

axis

Direction of an operation: axis=0 means rows (index), axis=1 means columns.

inplace

A parameter that, when True, changes the original object instead of returning a new one; deprecated for most methods from Pandas 3.1.

CSV file

A plain text file in which each line is a record and values are separated by a delimiter, usually a comma.

Delimiter

The character that separates values in a text file, set with sep in read_csv() and to_csv().

MCQs

Tap an option to check it, or use Show answers above to reveal them all.

1. Which Python library provides the Series and DataFrame data structures?

   1. NumPy
   2. Matplotlib
   3. Pandas
   4. SciPy
   Answer

   (c) Pandas

   Series and DataFrame are the two main data structures of Pandas. NumPy provides ndarray; Matplotlib is for plotting.

2. A Series is best described as:

   1. a one-dimensional labelled array of values of one data type
   2. a two-dimensional table with row and column labels
   3. a list of dictionaries
   4. a fixed-size array without labels
   Answer

   (a) a one-dimensional labelled array of values of one data type

   A Series is one-dimensional and has an index (labels). The table description fits a DataFrame.

3. Which statement creates a Series of four elements, each equal to 100?

   1. pd.Series(100, 4)
   2. pd.Series([100], index=4)
   3. pd.Series(100)
   4. pd.Series(100, index=['a', 'b', 'c', 'd'])
   Answer

   (d) pd.Series(100, index=['a', 'b', 'c', 'd'])

   A scalar value is repeated once for every label in index. pd.Series(100) makes only one element.

4. What is printed?

   print(len(pd.Series({'Goa': 2, 'Sikkim': 1, 'Bihar': 9})))

   1. 2
   2. 3
   3. 6
   4. 9
   Answer

   (b) 3

   The three keys become three index labels, so the Series has 3 elements.

5. s = pd.Series({'a': 1, 'b': 2}, index=['b', 'c']). What are the values of s?

   1. [1, 2]
   2. [2, nan]
   3. [2, 0]
   4. It raises KeyError
   Answer

   (b) [2, nan]

   Only the labels in index are used. 'b' gets 2 and 'c' is not a key, so it gets NaN.

6. s = pd.Series([3, 6, 9, 12]). What is printed by print(list(s.index))?

   1. [1, 2, 3, 4]
   2. [3, 6, 9, 12]
   3. [0, 1, 2, 3]
   4. ['0', '1', '2', '3']
   Answer

   (c) [0, 1, 2, 3]

   With no index given, Pandas numbers the elements from 0.

7. s = pd.Series([5, 10, 15, 20, 25], index=list('pqrst')). How many elements does s['q':'s'] return?

   1. 2
   2. 3
   3. 4
   4. 5
   Answer

   (b) 3

   A label slice includes both ends: q, r and s.

8. For the same Series s (labels p to t), which labels does s[1:3] return?

   1. q and r
   2. q, r and s
   3. p, q and r
   4. r and s
   Answer

   (a) q and r

   An integer slice works on positions and excludes the stop position 3, so positions 1 and 2 (q and r) are returned.

9. Which Series attribute gives the total number of elements, including NaN values?

   1. count
   2. shape[1]
   3. ndim
   4. size
   Answer

   (d) size

   size counts every element. The method count() skips NaN; ndim is always 1 for a Series; shape has only one item.

10. What is printed by print(pd.Series(dtype='float64').empty)?

    1. True
    2. False
    3. 0
    4. None
    Answer

    (a) True

    The Series has no elements, so empty is True.

11. When two Series are added with +, what appears for a label present in only one of them?

    1. 0
    2. The value from the Series that has it
    3. NaN
    4. An error is raised
    Answer

    (c) NaN

    Addition aligns on labels. A label missing from one side has nothing to add to, so the result is NaN. Use add(..., fill_value=0) to avoid it.

12. s1 = pd.Series([1, 2, 3], index=['x', 'y', 'z']) and s2 = pd.Series([4, 5], index=['y', 'z']). What is (s1 + s2)['z']?

    1. 7
    2. 8.0
    3. 5
    4. nan
    Answer

    (b) 8.0

    z is in both: 3 + 5 = 8. The result is float because label x gives NaN, which makes the whole Series float64.

13. s = pd.Series([3, 6, 9, 12, 15, 18]). What is printed by print(s.tail(2).tolist())?

    1. [3, 6]
    2. [15, 18]
    3. [12, 15, 18]
    4. [18]
    Answer

    (b) [15, 18]

    tail(2) returns the last two elements.

14. How many rows does df.head() return for a DataFrame with 50 rows?

    1. 1
    2. 3
    3. 10
    4. 5
    Answer

    (d) 5

    The default value of n in head() and tail() is 5.

15. Which works in every Pandas version to get the first element of a Series whose index labels are names of cities?

    1. s.iloc[0]
    2. s[0]
    3. s.loc[0]
    4. s.first
    Answer

    (a) s.iloc[0]

    iloc always uses positions. s[0] treats 0 as a label in Pandas 3 and raises KeyError; s.loc[0] always looks for label 0.

16. s = pd.Series([12, 45, 7, 60, 33]). What is printed by print(s[s > 30].tolist())?

    1. [45, 60]
    2. [45, 60, 33]
    3. [True, False]
    4. [1, 3, 4]
    Answer

    (b) [45, 60, 33]

    Boolean indexing keeps the values greater than 30, in their original order.

17. A DataFrame is created from a list of dictionaries. One dictionary has no key 'Phone' that the others have. What is stored in that cell?

    1. An empty string
    2. 0
    3. NaN
    4. The row is dropped
    Answer

    (c) NaN

    Missing keys become NaN in the DataFrame.

18. A DataFrame df has 4 rows and 3 columns. What is printed by print(df.shape, df.size)?

    1. (3, 4) 12
    2. (4, 3) 7
    3. 4 3
    4. (4, 3) 12
    Answer

    (d) (4, 3) 12

    shape is (rows, columns) and size is rows × columns = 12.

19. Which statement permanently removes the column 'Fees' from df?

    1. df.drop('Fees')
    2. df = df.drop('Fees', axis=0)
    3. df = df.drop('Fees', axis=1)
    4. df.remove('Fees')
    Answer

    (c) df = df.drop('Fees', axis=1)

    axis=1 points to columns, and assigning the result back makes the change permanent. The older form df.drop('Fees', axis=1, inplace=True) does the same. The first two options look for a row called Fees; there is no remove() method.

20. Which method lets you loop over a DataFrame row by row, getting the row label and the row as a Series?

    1. items()
    2. iterrows()
    3. iteritems()
    4. rows()
    Answer

    (b) iterrows()

    iterrows() yields (label, row) pairs. items() loops over columns; iteritems() was removed in Pandas 2.0.

21. What does df.T return?

    1. The transpose of df
    2. The total of every column
    3. The data types of df
    4. The last row of df
    Answer

    (a) The transpose of df

    T swaps rows and columns.

22. Which read_csv() parameter makes a column of the file the row labels of the DataFrame?

    1. header
    2. names
    3. index_col
    4. sep
    Answer

    (c) index_col

    index_col='RollNo' (or a column number) uses that column as the index.

23. Which to_csv() call writes the file without row labels?

    1. df.to_csv('a.csv', header=False)
    2. df.to_csv('a.csv', index=False)
    3. df.to_csv('a.csv', sep=None)
    4. df.to_csv('a.csv', names=False)
    Answer

    (b) df.to_csv('a.csv', index=False)

    index=False leaves out the row labels; header=False leaves out the column labels.

24. df has row labels 'R1' to 'R6'. How many rows does df.loc['R2':'R4'] return?

    1. 2
    2. 4
    3. 6
    4. 3
    Answer

    (d) 3

    loc slices include the end label: R2, R3 and R4.

25. Which statement renames the column 'Amt' to 'Amount' and leaves other columns unchanged?

    1. df.rename(columns={'Amt': 'Amount'})
    2. df.rename({'Amount': 'Amt'})
    3. df.columns = ['Amount']
    4. df.rename(index={'Amt': 'Amount'})
    Answer

    (a) df.rename(columns={'Amt': 'Amount'})

    The columns dictionary maps old label to new label. index= renames rows; assigning df.columns must list every column.

Assertion–Reason

1. Assertion (A): pd.Series([10, 20, 30], index=['a', 'b']) raises an error.

   Reason (R): When an index is given with a list of values, the number of labels must equal the number of values.

   Answer

   (A)

   Two labels cannot be matched to three values, so Pandas raises ValueError, exactly as R explains.

2. Assertion (A): For s = pd.Series([1, 2, 3, 4], index=['w', 'x', 'y', 'z']), s['x':'z'] returns three elements.

   Reason (R): Slicing a Series with labels excludes the stop label.

   Answer

   (C)

   A is true (x, y, z). R is false: label slices include the stop label; only position slices exclude it.

3. Assertion (A): Adding two Series that have different index labels can give a result with dtype float64 even if both Series hold integers.

   Reason (R): Labels that do not match produce NaN, and NaN is a floating-point value.

   Answer

   (A)

   The NaN values force the result to float64, which is the reason R gives.

4. Assertion (A): df.drop('Marks') removes the column Marks from df.

   Reason (R): By default drop() works along axis=0, that is, on row labels.

   Answer

   (D)

   A is false: with the default axis Pandas looks for a row called Marks and raises KeyError. R is true.

5. Assertion (A): df.size of a DataFrame with 30 rows and 5 columns is 150.

   Reason (R): size gives the number of rows of a DataFrame.

   Answer

   (C)

   A is true (30 × 5). R is false: size counts all cells; len(df) or df.shape[0] gives the number of rows.

6. Assertion (A): df.items() can be used to loop over the columns of a DataFrame.

   Reason (R): iterrows() returns each row as a Series.

   Answer

   (B)

   Both are true, but R describes a different method and does not explain A.

Short answer

1. 2 marks

   Give two differences between a Pandas Series and a NumPy ndarray.

   Answer

   1. A Series has an index of labels (which can be text), while an ndarray is accessed only by integer positions.
   2. When two Series are added, values are matched by label and unmatched labels give NaN; two ndarrays are added position by position and must have compatible shapes.
2. 2 marks

   Write Python code to create a Series fees holding 2500 for each of the months 'Apr', 'May' and 'Jun'.

   Answer

   A scalar is repeated for every label in the index.

   import pandas as pd
   fees = pd.Series(2500, index=['Apr', 'May', 'Jun'])
   print(fees)

   Output

   Apr    2500
   May    2500
   Jun    2500
   dtype: int64

3. 2 marks

   Write code to create a Series from the dictionary {'Mumbai': 12.4, 'Delhi': 16.8, 'Kolkata': 4.5} (population in millions) and print its index and values.

   Answer

   The keys become the index and the values become the data.

   import pandas as pd
   pop = pd.Series({'Mumbai': 12.4, 'Delhi': 16.8, 'Kolkata': 4.5})
   print(pop.index)
   print(pop.values)

   Output

   Index(['Mumbai', 'Delhi', 'Kolkata'], dtype='object')
   [12.4 16.8  4.5]

4. 2 marks

   Write the output:

   s = pd.Series([12, 24, 36, 48, 60], index=['p', 'q', 'r', 's', 't'])
print(s[1:4])
print(s['q':'s'])

   Answer

   Both give the same three elements: the position slice stops before position 4, and the label slice includes 's'.

   q    24
   r    36
   s    48
   dtype: int64
   q    24
   r    36
   s    48
   dtype: int64

5. 2 marks

   Write the output:

   s = pd.Series([4, 8, 12, 16], index=['a', 'b', 'c', 'd'])
print(s * 2 - 1)
print(s[s % 8 == 0])

   Answer

   Arithmetic with a scalar works on every element; the filter keeps multiples of 8.

   a     7
   b    15
   c    23
   d    31
   dtype: int64
   b     8
   d    16
   dtype: int64

6. 2 marks

   Write the output:

   a = pd.Series([10, 20, 30], index=['Mon', 'Tue', 'Wed'])
b = pd.Series([5, 15], index=['Tue', 'Thu'])
print(a + b)

   Answer

   Only Tue is in both Series. The result index is the sorted union of labels, and every unmatched label gives NaN.

   Mon     NaN
   Thu     NaN
   Tue    25.0
   Wed     NaN
   dtype: float64

7. 2 marks

   For the Series a and b of the previous question, write the output of print(a.add(b, fill_value=0)).

   Answer

   The missing side is taken as 0, so no NaN remains.

   Mon    10.0
   Thu    15.0
   Tue    25.0
   Wed    30.0
   dtype: float64

8. 2 marks

   Differentiate between loc and iloc with one example each.

   Answer

   loc selects by label, and a loc slice includes the end label: df.loc['S1':'S3', 'Maths'] gives rows S1, S2 and S3.

   iloc selects by integer position, and an iloc slice excludes the end position: df.iloc[0:2, 1] gives the first two rows of the second column.

9. 2 marks

   Write the output:

   s = pd.Series([90, 75, 82, 68, 95, 71], index=list('ABCDEF'))
print(s.head(2))
print(s.tail(-4))

   Answer

   head(2) gives the first two elements. A negative n in tail(-4) means all elements except the first 4.

   A    90
   B    75
   dtype: int64
   E    95
   F    71
   dtype: int64

10. 2 marks

    Name any four attributes of a DataFrame and state what each returns.

    Answer

    • index: row labels
    • columns: column labels
    • dtypes: data type of each column
    • shape: (rows, columns) as a tuple

    (Others: values, size, T, empty, ndim.)

11. 2 marks

    Write the output:

    df = pd.DataFrame({'Item': ['Pen', 'Ink', 'Pad'], 'Qty': [10, 4, 7]}, index=['i1', 'i2', 'i3'])
print(df.loc['i2':'i3', 'Qty'])
print(df.iloc[0, 0])

    Answer

    Selecting one column with a row slice gives a Series named Qty; iloc[0, 0] is the first cell.

    i2    4
    i3    7
    Name: Qty, dtype: int64
    Pen

12. 2 marks

    What is the difference between del df['Bonus'] and df.drop('Bonus', axis=1)?

    Answer

    del df['Bonus'] deletes the column from df itself and returns nothing; it works on one column only.

    df.drop('Bonus', axis=1) returns a new DataFrame without the column and leaves df unchanged unless the result is assigned back (or inplace=True is passed in older code). It can also remove several columns or rows at once.

13. 2 marks

    Write code to read stock.csv, which has no heading row and uses | as the separator, giving the columns the labels Code, Item and Qty.

    Answer

    Use header=None because the file has no heading row, and names to supply labels. For a file containing lines such as S01|Geometry box|40:

    import pandas as pd
    df = pd.read_csv('stock.csv', sep='|', header=None, names=['Code', 'Item', 'Qty'])
    print(df)

    Output

      Code           Item  Qty
    0  S01   Geometry box   40
    1  S02  Drawing sheet  250
    2  S03     Glue stick   75

14. 2 marks

    Why does an extra column such as Unnamed: 0 often appear when a DataFrame saved with to_csv() is read back? How do you prevent it?

    Answer

    By default to_csv() writes the row labels as the first column. When the file is read back, read_csv() treats it as an ordinary column with a blank heading, named Unnamed: 0.

    Prevent it by writing with df.to_csv('f.csv', index=False), or read it back with index_col=0.

15. 2 marks

    Write the output:

    df = pd.DataFrame([[1, 2], [3, 4], [5, 6]], columns=['x', 'y'])
df['z'] = df['x'] * df['y']
print(df.T)

    Answer

    Column z is added first; the transpose then turns the columns x, y, z into rows.

       0   1   2
    x  1   3   5
    y  2   4   6
    z  2  12  30

Long answer

1. 3 marks

   Create a Series marks from the NumPy array [67, 88, 94, 72, 55] with index ['Riya', 'Om', 'Liza', 'Jai', 'Esha']. Then write statements to:

   1. print the marks of Liza,
   2. print the first three students,
   3. print students who scored more than 70,
   4. add 5 grace marks to every student who scored below 60.
   Answer

   Use loc for labels, head() or a slice for the first three, and a Boolean condition both for filtering and for updating.

   import pandas as pd
   import numpy as np
   marks = pd.Series(np.array([67, 88, 94, 72, 55]), index=['Riya', 'Om', 'Liza', 'Jai', 'Esha'])
   print(marks.loc['Liza'])
   print(marks.head(3))
   print(marks[marks > 70])
   marks[marks < 60] = marks[marks < 60] + 5
   print(marks)

   Output

   94
   Riya    67
   Om      88
   Liza    94
   dtype: int64
   Om      88
   Liza    94
   Jai     72
   dtype: int64
   Riya    67
   Om      88
   Liza    94
   Jai     72
   Esha    60
   dtype: int64

2. 5 marks

   Explain with examples three ways of creating a DataFrame: from a dictionary of lists, from a list of dictionaries and from a dictionary of Series. Show how missing data is handled in each.

   Answer

   Dictionary of lists: keys become columns; all lists must be the same length, so there is no missing data (a short list raises ValueError).

   List of dictionaries: each dictionary is a row; a key missing in one dictionary gives NaN in that row.

   Dictionary of Series: each Series is a column and rows are matched by index labels; a label missing in one Series gives NaN.

   import pandas as pd
   d1 = pd.DataFrame({'City': ['Leh', 'Kochi'], 'Temp': [-4, 31]})
   d2 = pd.DataFrame([{'City': 'Leh', 'Temp': -4}, {'City': 'Kochi'}])
   d3 = pd.DataFrame({'Temp': pd.Series([-4, 31], index=['Leh', 'Kochi']),
                      'Rain': pd.Series([10], index=['Kochi'])})
   print(d1)
   print(d2)
   print(d3)

   Output

       City  Temp
   0    Leh    -4
   1  Kochi    31
       City  Temp
   0    Leh  -4.0
   1  Kochi   NaN
          Temp  Rain
   Kochi    31  10.0
   Leh      -4   NaN

3. 3 marks

   Consider df = pd.read_csv('marks.csv', index_col='RollNo') where the file holds columns RollNo, Name, Maths, Science and English for five students. Write statements to:

   1. display the names of students with Maths above 80,
   2. add a column Total,
   3. display rows of roll numbers 2 to 4 with only Name and Total,
   4. delete the column English.
   Answer

   loc with a Boolean condition selects the names; the new column is a sum of three columns; a loc slice on roll numbers includes roll 4; drop(columns=...) removes English.

   import pandas as pd
   df = pd.read_csv('marks.csv', index_col='RollNo')
   print(df.loc[df['Maths'] > 80, 'Name'])
   df['Total'] = df['Maths'] + df['Science'] + df['English']
   print(df.loc[2:4, ['Name', 'Total']])
   df = df.drop(columns='English')
   print(df.columns.tolist())

   Output

   RollNo
   2     Diya
   4    Meera
   Name: Name, dtype: object
            Name  Total
   RollNo              
   2        Diya    274
   3       Kabir    195
   4       Meera    252
   ['Name', 'Maths', 'Science', 'Total']

4. 3 marks

   What is index alignment? Explain with an example how + and add() with fill_value behave differently for two Series of runs scored by players in two matches.

   Answer

   When two Series are combined, Pandas pairs values that have the same index label, whatever their positions. This is called alignment. With +, a player who batted in only one match has nothing to pair with, so the result is NaN. add(other, fill_value=0) treats the missing score as 0, so the total equals the one score.

   import pandas as pd
   m1 = pd.Series([56, 12, 80], index=['Gill', 'Iyer', 'Pant'])
   m2 = pd.Series([34, 101], index=['Pant', 'Rahul'])
   print(m1 + m2)
   print(m1.add(m2, fill_value=0))

   Output

   Gill       NaN
   Iyer       NaN
   Pant     114.0
   Rahul      NaN
   dtype: float64
   Gill      56.0
   Iyer      12.0
   Pant     114.0
   Rahul    101.0
   dtype: float64

5. 5 marks

   Explain read_csv() and to_csv() with the parameters sep, header, names, index_col and index. Write code that reads a semicolon-separated file stations.txt (no heading row; columns: code, station, state, platforms), keeps stations with more than 15 platforms and saves them to big.csv without row labels.

   Answer

   • sep: separator character (',' by default).
   • header: row number holding the labels in read_csv(); None if there is none. In to_csv(), header=False skips writing column labels.
   • names: list of column labels to use while reading.
   • index_col: column to use as row labels while reading.
   • index: in to_csv(), whether to write row labels (True by default).

   import pandas as pd
   st = pd.read_csv('stations.txt', sep=';', header=None,
                    names=['Code', 'Station', 'State', 'Platforms'])
   big = st[st['Platforms'] > 15]
   big.to_csv('big.csv', index=False)
   print(open('big.csv').read())

   Output

   Code,Station,State,Platforms
   NDLS,New Delhi,Delhi,16
   MAS,Chennai Central,Tamil Nadu,17
   HWH,Howrah,West Bengal,23

6. 3 marks

   Write a program that creates a DataFrame of three trains (TrainNo, Name, Coaches) and then: adds a new train using loc, renames the column Coaches to Cars, removes the train at row label 1 and prints each remaining train with iterrows().

   Answer

   loc[len(tr)] adds a row at the end; rename() and drop() return new DataFrames, so their results are assigned back; row label 1 (Mumbai Rajdhani) is dropped and the labels 0, 2, 3 remain.

   import pandas as pd
   tr = pd.DataFrame({'TrainNo': [12002, 12951, 22436],
                      'Name': ['Shatabdi', 'Rajdhani', 'Vande Bharat'],
                      'Coaches': [18, 22, 16]})
   tr.loc[len(tr)] = [12627, 'Karnataka Exp', 24]
   tr = tr.rename(columns={'Coaches': 'Cars'})
   tr = tr.drop(1)
   for label, row in tr.iterrows():
       print(label, row['TrainNo'], row['Name'], row['Cars'])

   Output

   0 12002 Shatabdi 18
   2 22436 Vande Bharat 16
   3 12627 Karnataka Exp 24

7. 3 marks

   Write the output of the following code and explain each line.

   s = pd.Series([2, 4, 6, 8, 10], index=[10, 20, 30, 40, 50])
print(s[20])
print(s.iloc[2])
print(s.loc[20:40].sum())
print(s.iloc[1:3].tolist())

   Answer

   The index is made of integers, so s[20] is a label lookup and gives 4. s.iloc[2] is the third element, 6. s.loc[20:40] includes labels 20, 30 and 40, whose sum is 4 + 6 + 8 = 18. s.iloc[1:3] takes positions 1 and 2.

   4
   6
   18
   [4, 6]

Case-based

Practicals

import pandas as pd
import numpy as np
capital = pd.Series({'Rajasthan': 'Jaipur', 'Odisha': 'Bhubaneswar',
                     'Kerala': 'Thiruvananthapuram', 'Manipur': 'Imphal'})
area = pd.Series(np.array([342, 156, 39, 22]), index=capital.index)
print(capital)
print(area)
print(capital.dtype, area.dtype)
print('Second state:', area.index[1], area.iloc[1])

Rajasthan                Jaipur
Odisha              Bhubaneswar
Kerala       Thiruvananthapuram
Manipur                  Imphal
dtype: object
Rajasthan    342
Odisha       156
Kerala        39
Manipur       22
dtype: int64
object int64
Second state: Odisha 156

import pandas as pd
bus_fee = pd.Series(1800, index=['Apr', 'May', 'Jun', 'Jul', 'Aug'])
bus_fee.name = 'BusFee'
bus_fee.index.name = 'Month'
print(bus_fee)
print(bus_fee.index)
print(bus_fee.values)
print(bus_fee.shape, bus_fee.size, bus_fee.ndim, bus_fee.empty, bus_fee.name)

Month
Apr    1800
May    1800
Jun    1800
Jul    1800
Aug    1800
Name: BusFee, dtype: int64
Index(['Apr', 'May', 'Jun', 'Jul', 'Aug'], dtype='object', name='Month')
[1800 1800 1800 1800 1800]
(5,) 5 1 False BusFee

import pandas as pd
inn1 = pd.Series([88, 12, 45, 131], index=['Jaiswal', 'Gill', 'Rahul', 'Pant'])
inn2 = pd.Series([54, 102, 9], index=['Jaiswal', 'Gill', 'Jadeja'])
print(inn1 + inn2)
total = inn1.add(inn2, fill_value=0)
print(total)
print(inn1.sub(inn2, fill_value=0))
print(total[total >= 100])

Gill       114.0
Jadeja       NaN
Jaiswal    142.0
Pant         NaN
Rahul        NaN
dtype: float64
Gill       114.0
Jadeja       9.0
Jaiswal    142.0
Pant       131.0
Rahul       45.0
dtype: float64
Gill       -90.0
Jadeja      -9.0
Jaiswal     34.0
Pant       131.0
Rahul       45.0
dtype: float64
Gill       114.0
Jaiswal    142.0
Pant       131.0
dtype: float64

import pandas as pd
marks = pd.Series([56, 78, 91, 45, 67, 88, 72, 95, 60, 83],
                  index=['Asha', 'Bilal', 'Charu', 'Dhruv', 'Ekta',
                         'Faiz', 'Gauri', 'Harsh', 'Indu', 'Jatin'])
q3 = marks.quantile(0.75)
print('75th percentile:', q3)
print(marks[marks > q3])

75th percentile: 86.75
Charu    91
Faiz     88
Harsh    95
dtype: int64

import pandas as pd
result = pd.DataFrame({'Name': ['Ishaan', 'Tanvi', 'Farhan', 'Sneha', 'Yash', 'Pooja'],
                       'Class': ['12A', '12B', '12A', '12C', '12B', '12A'],
                       'Percent': [86.4, 92.0, 74.5, 81.2, 68.9, 95.6]},
                      index=['A101', 'A102', 'A103', 'A104', 'A105', 'A106'])
print(result)
print('Row labels:', result.index.tolist())
print('Column labels:', result.columns.tolist())
print(result.dtypes)
print('Dimensions:', result.ndim, 'Shape:', result.shape, 'Size:', result.size)

        Name Class  Percent
A101  Ishaan   12A     86.4
A102   Tanvi   12B     92.0
A103  Farhan   12A     74.5
A104   Sneha   12C     81.2
A105    Yash   12B     68.9
A106   Pooja   12A     95.6
Row labels: ['A101', 'A102', 'A103', 'A104', 'A105', 'A106']
Column labels: ['Name', 'Class', 'Percent']
Name        object
Class       object
Percent    float64
dtype: object
Dimensions: 2 Shape: (6, 3) Size: 18

import pandas as pd
lib = pd.DataFrame({'RollNo': [11, 14, 11, 20, 14, 11],
                    'Name': ['Nisha', 'Aman', 'Nisha', 'Ritu', 'Aman', 'Nisha'],
                    'Book': ['Wings of Fire', 'Godan', 'Wings of Fire',
                             'Malgudi Days', 'Wings of Fire', 'Gitanjali']})
print(lib[lib.duplicated()])
clean = lib.drop_duplicates()
print(clean)
print(clean[clean['Book'] == 'Wings of Fire'])
print(clean[clean.duplicated(subset=['RollNo'])]['Name'].unique())

   RollNo   Name           Book
2      11  Nisha  Wings of Fire
   RollNo   Name           Book
0      11  Nisha  Wings of Fire
1      14   Aman          Godan
3      20   Ritu   Malgudi Days
4      14   Aman  Wings of Fire
5      11  Nisha      Gitanjali
   RollNo   Name           Book
0      11  Nisha  Wings of Fire
4      14   Aman  Wings of Fire
['Aman' 'Nisha']

import pandas as pd
df = pd.read_csv('marks.csv', index_col='RollNo')
df['Percent'] = ((df['Maths'] + df['Science'] + df['English']) / 3).round(1)
merit = df[df['Percent'] >= 75]
merit.to_csv('merit.csv', sep=';')
print(open('merit.csv').read())
back = pd.read_csv('merit.csv', sep=';', index_col='RollNo')
print(back)

RollNo;Name;Maths;Science;English;Percent
1;Aarav;78;82;74;78.0
2;Diya;91;88;95;91.3
4;Meera;88;79;85;84.0

         Name  Maths  Science  English  Percent
RollNo                                         
1       Aarav     78       82       74     78.0
2        Diya     91       88       95     91.3
4       Meera     88       79       85     84.0

import pandas as pd
st = pd.DataFrame({'Capital': ['Jaipur', 'Patna', 'Shimla', 'Panaji'],
                   'Districts': [50, 38, 12, 2]},
                  index=['Rajasthan', 'Bihar', 'Himachal', 'Goa'])
st['Zone'] = ['North', 'East', 'North', 'West']
print(st)
st.loc['Tripura'] = ['Agartala', 8, 'North-East']
print(st)
st = st.rename(columns={'Districts': 'NumDistricts'})
del st['Zone']
st = st.drop('Bihar')
print(st)

          Capital  Districts   Zone
Rajasthan  Jaipur         50  North
Bihar       Patna         38   East
Himachal   Shimla         12  North
Goa        Panaji          2   West
            Capital  Districts        Zone
Rajasthan    Jaipur         50       North
Bihar         Patna         38        East
Himachal     Shimla         12       North
Goa          Panaji          2        West
Tripura    Agartala          8  North-East
            Capital  NumDistricts
Rajasthan    Jaipur            50
Himachal     Shimla            12
Goa          Panaji             2
Tripura    Agartala             8

import pandas as pd
df = pd.DataFrame({'Maths': [78, 30, 65, 88, 55],
                   'Science': [82, 62, 28, 79, 62],
                   'English': [74, 70, 60, 85, 58]},
                  index=['Aarav', 'Diya', 'Kabir', 'Meera', 'Rohan'])
for name, row in df.iterrows():
    remark = 'PASS' if row.min() >= 33 else 'FAIL'
    print(f'{name:<6} Total={row.sum():>3} {remark}')
for subject, col in df.items():
    print(subject, 'highest:', col.max())

Aarav  Total=234 PASS
Diya   Total=162 FAIL
Kabir  Total=153 FAIL
Meera  Total=252 PASS
Rohan  Total=175 PASS
Maths highest: 88
Science highest: 82
English highest: 85

import pandas as pd
tr = pd.DataFrame({'Name': ['Howrah Rajdhani', 'Mumbai Rajdhani', 'Shatabdi', 'Vande Bharat'],
                   'From': ['New Delhi', 'New Delhi', 'Chennai', 'Varanasi'],
                   'To': ['Howrah', 'Mumbai', 'Mysuru', 'New Delhi'],
                   'Departure': ['16:50', '16:55', '06:00', '15:00']},
                  index=[12301, 12951, 12007, 22435])
tr.index.name = 'TrainNo'
print(tr.loc[12951])
print(tr.iloc[0:3, [0, 3]])
print(tr.loc[12301:12951])
print(tr.iloc[-1]['Departure'])

Name         Mumbai Rajdhani
From               New Delhi
To                    Mumbai
Departure              16:55
Name: 12951, dtype: object
                    Name Departure
TrainNo                           
12301    Howrah Rajdhani     16:50
12951    Mumbai Rajdhani     16:55
12007           Shatabdi     06:00
                    Name       From      To Departure
TrainNo                                              
12301    Howrah Rajdhani  New Delhi  Howrah     16:50
12951    Mumbai Rajdhani  New Delhi  Mumbai     16:55
15:00

Common mistakes

• Writing pd.series(...) or pd.dataframe(...). Python is case-sensitive: the correct names are pd.Series and pd.DataFrame.

• Creating a Series from a scalar without an index, expecting many elements. pd.Series(5) has one element; write pd.Series(5, index=[...]).

• Expecting a label slice to stop before the end label. s['b':'d'] includes 'd'; only position slices such as s[1:3] exclude the end.

• Using s[0] to get the first element of a Series whose labels are words or roll numbers. It raises KeyError in Pandas 3 (and a warning in 2.x). Use s.iloc[0].

• Adding two Series and expecting position-wise results. Pandas matches labels, so unmatched labels give NaN; use add(other, fill_value=0) when a missing value should count as 0.

• Writing df[df.Maths > 80 and df.Science > 80]. Use & instead of and, and put brackets around each condition: df[(df.Maths > 80) & (df.Science > 80)].

• Calling df.drop('Grade') to remove a column. Without axis=1 (or columns=), Pandas looks for a row called Grade and raises KeyError.

• Forgetting that drop() and rename() return a new DataFrame. Assign the result back (df = df.drop(...)). Never combine the two styles: df = df.drop(..., inplace=True) sets df to None.

• Using DataFrame.append() or iteritems() from old books. Both were removed in Pandas 2.0; use pd.concat() / df.loc[new_label] = ... and items().

• Exporting with to_csv('f.csv') and then finding an extra unnamed column when reading back. Pass index=False when the row labels are just 0, 1, 2, ...