Pandas reset inner level of MultiIndex
Clash Royale CLAN TAG#URR8PPP
up vote
9
down vote
favorite
I have a DF in the following format:
col1 col2
ID Date
1 1993-12-31 4 6
1994-12-31 8 5
1995-12-31 4 7
1996-12-31 3 3
2 2000-12-31 7 8
2001-12-31 5 9
2002-12-31 8 4
And I want to reset the 'Date' index giving the following:
col1 col2
ID Date
1 0 4 6
1 8 5
2 4 7
3 3 3
2 0 7 8
1 5 9
2 8 4
I thought simply df.reset_index(level='Date', inplace=True, drop=True)
would do it, but it does not.
python pandas
add a comment |Â
up vote
9
down vote
favorite
I have a DF in the following format:
col1 col2
ID Date
1 1993-12-31 4 6
1994-12-31 8 5
1995-12-31 4 7
1996-12-31 3 3
2 2000-12-31 7 8
2001-12-31 5 9
2002-12-31 8 4
And I want to reset the 'Date' index giving the following:
col1 col2
ID Date
1 0 4 6
1 8 5
2 4 7
3 3 3
2 0 7 8
1 5 9
2 8 4
I thought simply df.reset_index(level='Date', inplace=True, drop=True)
would do it, but it does not.
python pandas
add a comment |Â
up vote
9
down vote
favorite
up vote
9
down vote
favorite
I have a DF in the following format:
col1 col2
ID Date
1 1993-12-31 4 6
1994-12-31 8 5
1995-12-31 4 7
1996-12-31 3 3
2 2000-12-31 7 8
2001-12-31 5 9
2002-12-31 8 4
And I want to reset the 'Date' index giving the following:
col1 col2
ID Date
1 0 4 6
1 8 5
2 4 7
3 3 3
2 0 7 8
1 5 9
2 8 4
I thought simply df.reset_index(level='Date', inplace=True, drop=True)
would do it, but it does not.
python pandas
I have a DF in the following format:
col1 col2
ID Date
1 1993-12-31 4 6
1994-12-31 8 5
1995-12-31 4 7
1996-12-31 3 3
2 2000-12-31 7 8
2001-12-31 5 9
2002-12-31 8 4
And I want to reset the 'Date' index giving the following:
col1 col2
ID Date
1 0 4 6
1 8 5
2 4 7
3 3 3
2 0 7 8
1 5 9
2 8 4
I thought simply df.reset_index(level='Date', inplace=True, drop=True)
would do it, but it does not.
python pandas
python pandas
asked Aug 13 at 18:31
KOB
1,011617
1,011617
add a comment |Â
add a comment |Â
4 Answers
4
active
oldest
votes
up vote
4
down vote
accepted
Using pd.MultiIndex.from_arrays
and groupby
+ cumcount
.
df.index = pd.MultiIndex.from_arrays(
[df.index.get_level_values(0), df.groupby(level=0).cumcount()],
names=['ID', 'Date'])
df
col1 col2
ID Date
1 0 4 6
1 8 5
2 4 7
3 3 3
2 0 7 8
1 5 9
2 8 4
This won't generalise to N levels, but there should be a df.index.set_levels
equivalent I'm forgetting...
2
I find this solution easiest to follow.
â KOB
Aug 13 at 18:52
add a comment |Â
up vote
4
down vote
You can groupby ID
, then reset the index on each group using apply
:
new_df = (df.groupby(df.index.get_level_values('ID'))
.apply(lambda x: x.reset_index()).drop(['ID','Date'],1))
new_df.index = new_df.index.rename(['ID','Date'])
>>> new_df
col1 col2
ID Date
1 0 4 6
1 8 5
2 4 7
3 3 3
2 0 7 8
1 5 9
2 8 4
add a comment |Â
up vote
4
down vote
Using set_index
and cumcount
:
tmp = df.reset_index('Date', drop=True)
tmp.set_index(df.groupby(level=0).cumcount().rename('Date'), append=True)
col1 col2
ID Date
1 0 4 6
1 8 5
2 4 7
3 3 3
2 0 7 8
1 5 9
2 8 4
add a comment |Â
up vote
3
down vote
New Answer
Not as cool as the old answer but I'd rather be accurate than cool.
from collections import defaultdict
from itertools import count
d = defaultdict(count)
lbl =
for a, *_ in df.index.values:
lbl.append(next(d[a]))
lvl = pd.RangeIndex(max(lbl) + 1)
df.set_index(df.index.set_labels(lbl, 1).set_levels(lvl, 1))
col1 col2
ID Date
1 0 4 6
1 8 5
2 4 7
3 3 3
2 0 7 8
1 5 9
2 8 4
OLD ANSWER
Do Not Use
I misread the question. I didn't see that the new index needed to reset for every group.
Hopefully useful to someone.
You can use pandas.MultiIndex.set_levels
n = 1
lvl = df.index.levels[n]
new_lvl = pd.RangeIndex(len(lvl))
new_idx = df.index.set_levels(new_lvl, n)
df.set_index(new_idx)
col1 col2
ID Date
1 0 4 6
1 8 5
2 4 7
3 3 3
2 4 7 8
5 5 9
6 8 4
One-line
Yay! o/
df.set_index(df.index.set_levels(pd.RangeIndex(len(df.index.levels[1])), 1))
col1 col2
ID Date
1 0 4 6
1 8 5
2 4 7
3 3 3
2 4 7 8
5 5 9
6 8 4
In place
df.index.set_levels(pd.RangeIndex(len(df.index.levels[1])), 1, inplace=True)
df
col1 col2
ID Date
1 0 4 6
1 8 5
2 4 7
3 3 3
2 4 7 8
5 5 9
6 8 4
add a comment |Â
4 Answers
4
active
oldest
votes
4 Answers
4
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
4
down vote
accepted
Using pd.MultiIndex.from_arrays
and groupby
+ cumcount
.
df.index = pd.MultiIndex.from_arrays(
[df.index.get_level_values(0), df.groupby(level=0).cumcount()],
names=['ID', 'Date'])
df
col1 col2
ID Date
1 0 4 6
1 8 5
2 4 7
3 3 3
2 0 7 8
1 5 9
2 8 4
This won't generalise to N levels, but there should be a df.index.set_levels
equivalent I'm forgetting...
2
I find this solution easiest to follow.
â KOB
Aug 13 at 18:52
add a comment |Â
up vote
4
down vote
accepted
Using pd.MultiIndex.from_arrays
and groupby
+ cumcount
.
df.index = pd.MultiIndex.from_arrays(
[df.index.get_level_values(0), df.groupby(level=0).cumcount()],
names=['ID', 'Date'])
df
col1 col2
ID Date
1 0 4 6
1 8 5
2 4 7
3 3 3
2 0 7 8
1 5 9
2 8 4
This won't generalise to N levels, but there should be a df.index.set_levels
equivalent I'm forgetting...
2
I find this solution easiest to follow.
â KOB
Aug 13 at 18:52
add a comment |Â
up vote
4
down vote
accepted
up vote
4
down vote
accepted
Using pd.MultiIndex.from_arrays
and groupby
+ cumcount
.
df.index = pd.MultiIndex.from_arrays(
[df.index.get_level_values(0), df.groupby(level=0).cumcount()],
names=['ID', 'Date'])
df
col1 col2
ID Date
1 0 4 6
1 8 5
2 4 7
3 3 3
2 0 7 8
1 5 9
2 8 4
This won't generalise to N levels, but there should be a df.index.set_levels
equivalent I'm forgetting...
Using pd.MultiIndex.from_arrays
and groupby
+ cumcount
.
df.index = pd.MultiIndex.from_arrays(
[df.index.get_level_values(0), df.groupby(level=0).cumcount()],
names=['ID', 'Date'])
df
col1 col2
ID Date
1 0 4 6
1 8 5
2 4 7
3 3 3
2 0 7 8
1 5 9
2 8 4
This won't generalise to N levels, but there should be a df.index.set_levels
equivalent I'm forgetting...
answered Aug 13 at 18:49
coldspeed
104k1789157
104k1789157
2
I find this solution easiest to follow.
â KOB
Aug 13 at 18:52
add a comment |Â
2
I find this solution easiest to follow.
â KOB
Aug 13 at 18:52
2
2
I find this solution easiest to follow.
â KOB
Aug 13 at 18:52
I find this solution easiest to follow.
â KOB
Aug 13 at 18:52
add a comment |Â
up vote
4
down vote
You can groupby ID
, then reset the index on each group using apply
:
new_df = (df.groupby(df.index.get_level_values('ID'))
.apply(lambda x: x.reset_index()).drop(['ID','Date'],1))
new_df.index = new_df.index.rename(['ID','Date'])
>>> new_df
col1 col2
ID Date
1 0 4 6
1 8 5
2 4 7
3 3 3
2 0 7 8
1 5 9
2 8 4
add a comment |Â
up vote
4
down vote
You can groupby ID
, then reset the index on each group using apply
:
new_df = (df.groupby(df.index.get_level_values('ID'))
.apply(lambda x: x.reset_index()).drop(['ID','Date'],1))
new_df.index = new_df.index.rename(['ID','Date'])
>>> new_df
col1 col2
ID Date
1 0 4 6
1 8 5
2 4 7
3 3 3
2 0 7 8
1 5 9
2 8 4
add a comment |Â
up vote
4
down vote
up vote
4
down vote
You can groupby ID
, then reset the index on each group using apply
:
new_df = (df.groupby(df.index.get_level_values('ID'))
.apply(lambda x: x.reset_index()).drop(['ID','Date'],1))
new_df.index = new_df.index.rename(['ID','Date'])
>>> new_df
col1 col2
ID Date
1 0 4 6
1 8 5
2 4 7
3 3 3
2 0 7 8
1 5 9
2 8 4
You can groupby ID
, then reset the index on each group using apply
:
new_df = (df.groupby(df.index.get_level_values('ID'))
.apply(lambda x: x.reset_index()).drop(['ID','Date'],1))
new_df.index = new_df.index.rename(['ID','Date'])
>>> new_df
col1 col2
ID Date
1 0 4 6
1 8 5
2 4 7
3 3 3
2 0 7 8
1 5 9
2 8 4
answered Aug 13 at 18:37
sacul
18.4k31434
18.4k31434
add a comment |Â
add a comment |Â
up vote
4
down vote
Using set_index
and cumcount
:
tmp = df.reset_index('Date', drop=True)
tmp.set_index(df.groupby(level=0).cumcount().rename('Date'), append=True)
col1 col2
ID Date
1 0 4 6
1 8 5
2 4 7
3 3 3
2 0 7 8
1 5 9
2 8 4
add a comment |Â
up vote
4
down vote
Using set_index
and cumcount
:
tmp = df.reset_index('Date', drop=True)
tmp.set_index(df.groupby(level=0).cumcount().rename('Date'), append=True)
col1 col2
ID Date
1 0 4 6
1 8 5
2 4 7
3 3 3
2 0 7 8
1 5 9
2 8 4
add a comment |Â
up vote
4
down vote
up vote
4
down vote
Using set_index
and cumcount
:
tmp = df.reset_index('Date', drop=True)
tmp.set_index(df.groupby(level=0).cumcount().rename('Date'), append=True)
col1 col2
ID Date
1 0 4 6
1 8 5
2 4 7
3 3 3
2 0 7 8
1 5 9
2 8 4
Using set_index
and cumcount
:
tmp = df.reset_index('Date', drop=True)
tmp.set_index(df.groupby(level=0).cumcount().rename('Date'), append=True)
col1 col2
ID Date
1 0 4 6
1 8 5
2 4 7
3 3 3
2 0 7 8
1 5 9
2 8 4
edited Aug 13 at 19:07
answered Aug 13 at 18:48
user3483203
24.1k62147
24.1k62147
add a comment |Â
add a comment |Â
up vote
3
down vote
New Answer
Not as cool as the old answer but I'd rather be accurate than cool.
from collections import defaultdict
from itertools import count
d = defaultdict(count)
lbl =
for a, *_ in df.index.values:
lbl.append(next(d[a]))
lvl = pd.RangeIndex(max(lbl) + 1)
df.set_index(df.index.set_labels(lbl, 1).set_levels(lvl, 1))
col1 col2
ID Date
1 0 4 6
1 8 5
2 4 7
3 3 3
2 0 7 8
1 5 9
2 8 4
OLD ANSWER
Do Not Use
I misread the question. I didn't see that the new index needed to reset for every group.
Hopefully useful to someone.
You can use pandas.MultiIndex.set_levels
n = 1
lvl = df.index.levels[n]
new_lvl = pd.RangeIndex(len(lvl))
new_idx = df.index.set_levels(new_lvl, n)
df.set_index(new_idx)
col1 col2
ID Date
1 0 4 6
1 8 5
2 4 7
3 3 3
2 4 7 8
5 5 9
6 8 4
One-line
Yay! o/
df.set_index(df.index.set_levels(pd.RangeIndex(len(df.index.levels[1])), 1))
col1 col2
ID Date
1 0 4 6
1 8 5
2 4 7
3 3 3
2 4 7 8
5 5 9
6 8 4
In place
df.index.set_levels(pd.RangeIndex(len(df.index.levels[1])), 1, inplace=True)
df
col1 col2
ID Date
1 0 4 6
1 8 5
2 4 7
3 3 3
2 4 7 8
5 5 9
6 8 4
add a comment |Â
up vote
3
down vote
New Answer
Not as cool as the old answer but I'd rather be accurate than cool.
from collections import defaultdict
from itertools import count
d = defaultdict(count)
lbl =
for a, *_ in df.index.values:
lbl.append(next(d[a]))
lvl = pd.RangeIndex(max(lbl) + 1)
df.set_index(df.index.set_labels(lbl, 1).set_levels(lvl, 1))
col1 col2
ID Date
1 0 4 6
1 8 5
2 4 7
3 3 3
2 0 7 8
1 5 9
2 8 4
OLD ANSWER
Do Not Use
I misread the question. I didn't see that the new index needed to reset for every group.
Hopefully useful to someone.
You can use pandas.MultiIndex.set_levels
n = 1
lvl = df.index.levels[n]
new_lvl = pd.RangeIndex(len(lvl))
new_idx = df.index.set_levels(new_lvl, n)
df.set_index(new_idx)
col1 col2
ID Date
1 0 4 6
1 8 5
2 4 7
3 3 3
2 4 7 8
5 5 9
6 8 4
One-line
Yay! o/
df.set_index(df.index.set_levels(pd.RangeIndex(len(df.index.levels[1])), 1))
col1 col2
ID Date
1 0 4 6
1 8 5
2 4 7
3 3 3
2 4 7 8
5 5 9
6 8 4
In place
df.index.set_levels(pd.RangeIndex(len(df.index.levels[1])), 1, inplace=True)
df
col1 col2
ID Date
1 0 4 6
1 8 5
2 4 7
3 3 3
2 4 7 8
5 5 9
6 8 4
add a comment |Â
up vote
3
down vote
up vote
3
down vote
New Answer
Not as cool as the old answer but I'd rather be accurate than cool.
from collections import defaultdict
from itertools import count
d = defaultdict(count)
lbl =
for a, *_ in df.index.values:
lbl.append(next(d[a]))
lvl = pd.RangeIndex(max(lbl) + 1)
df.set_index(df.index.set_labels(lbl, 1).set_levels(lvl, 1))
col1 col2
ID Date
1 0 4 6
1 8 5
2 4 7
3 3 3
2 0 7 8
1 5 9
2 8 4
OLD ANSWER
Do Not Use
I misread the question. I didn't see that the new index needed to reset for every group.
Hopefully useful to someone.
You can use pandas.MultiIndex.set_levels
n = 1
lvl = df.index.levels[n]
new_lvl = pd.RangeIndex(len(lvl))
new_idx = df.index.set_levels(new_lvl, n)
df.set_index(new_idx)
col1 col2
ID Date
1 0 4 6
1 8 5
2 4 7
3 3 3
2 4 7 8
5 5 9
6 8 4
One-line
Yay! o/
df.set_index(df.index.set_levels(pd.RangeIndex(len(df.index.levels[1])), 1))
col1 col2
ID Date
1 0 4 6
1 8 5
2 4 7
3 3 3
2 4 7 8
5 5 9
6 8 4
In place
df.index.set_levels(pd.RangeIndex(len(df.index.levels[1])), 1, inplace=True)
df
col1 col2
ID Date
1 0 4 6
1 8 5
2 4 7
3 3 3
2 4 7 8
5 5 9
6 8 4
New Answer
Not as cool as the old answer but I'd rather be accurate than cool.
from collections import defaultdict
from itertools import count
d = defaultdict(count)
lbl =
for a, *_ in df.index.values:
lbl.append(next(d[a]))
lvl = pd.RangeIndex(max(lbl) + 1)
df.set_index(df.index.set_labels(lbl, 1).set_levels(lvl, 1))
col1 col2
ID Date
1 0 4 6
1 8 5
2 4 7
3 3 3
2 0 7 8
1 5 9
2 8 4
OLD ANSWER
Do Not Use
I misread the question. I didn't see that the new index needed to reset for every group.
Hopefully useful to someone.
You can use pandas.MultiIndex.set_levels
n = 1
lvl = df.index.levels[n]
new_lvl = pd.RangeIndex(len(lvl))
new_idx = df.index.set_levels(new_lvl, n)
df.set_index(new_idx)
col1 col2
ID Date
1 0 4 6
1 8 5
2 4 7
3 3 3
2 4 7 8
5 5 9
6 8 4
One-line
Yay! o/
df.set_index(df.index.set_levels(pd.RangeIndex(len(df.index.levels[1])), 1))
col1 col2
ID Date
1 0 4 6
1 8 5
2 4 7
3 3 3
2 4 7 8
5 5 9
6 8 4
In place
df.index.set_levels(pd.RangeIndex(len(df.index.levels[1])), 1, inplace=True)
df
col1 col2
ID Date
1 0 4 6
1 8 5
2 4 7
3 3 3
2 4 7 8
5 5 9
6 8 4
edited Aug 13 at 19:36
answered Aug 13 at 19:05
piRSquared
140k19118248
140k19118248
add a comment |Â
add a comment |Â
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f51828518%2fpandas-reset-inner-level-of-multiindex%23new-answer', 'question_page');
);
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password