Pandas reset inner level of MultiIndex

The name of the pictureThe name of the pictureThe name of the pictureClash Royale CLAN TAG#URR8PPP











up vote
9
down vote

favorite












I have a DF in the following format:



 col1 col2
ID Date
1 1993-12-31 4 6
1994-12-31 8 5
1995-12-31 4 7
1996-12-31 3 3
2 2000-12-31 7 8
2001-12-31 5 9
2002-12-31 8 4


And I want to reset the 'Date' index giving the following:



 col1 col2
ID Date
1 0 4 6
1 8 5
2 4 7
3 3 3
2 0 7 8
1 5 9
2 8 4


I thought simply df.reset_index(level='Date', inplace=True, drop=True) would do it, but it does not.










share|improve this question

























    up vote
    9
    down vote

    favorite












    I have a DF in the following format:



     col1 col2
    ID Date
    1 1993-12-31 4 6
    1994-12-31 8 5
    1995-12-31 4 7
    1996-12-31 3 3
    2 2000-12-31 7 8
    2001-12-31 5 9
    2002-12-31 8 4


    And I want to reset the 'Date' index giving the following:



     col1 col2
    ID Date
    1 0 4 6
    1 8 5
    2 4 7
    3 3 3
    2 0 7 8
    1 5 9
    2 8 4


    I thought simply df.reset_index(level='Date', inplace=True, drop=True) would do it, but it does not.










    share|improve this question























      up vote
      9
      down vote

      favorite









      up vote
      9
      down vote

      favorite











      I have a DF in the following format:



       col1 col2
      ID Date
      1 1993-12-31 4 6
      1994-12-31 8 5
      1995-12-31 4 7
      1996-12-31 3 3
      2 2000-12-31 7 8
      2001-12-31 5 9
      2002-12-31 8 4


      And I want to reset the 'Date' index giving the following:



       col1 col2
      ID Date
      1 0 4 6
      1 8 5
      2 4 7
      3 3 3
      2 0 7 8
      1 5 9
      2 8 4


      I thought simply df.reset_index(level='Date', inplace=True, drop=True) would do it, but it does not.










      share|improve this question













      I have a DF in the following format:



       col1 col2
      ID Date
      1 1993-12-31 4 6
      1994-12-31 8 5
      1995-12-31 4 7
      1996-12-31 3 3
      2 2000-12-31 7 8
      2001-12-31 5 9
      2002-12-31 8 4


      And I want to reset the 'Date' index giving the following:



       col1 col2
      ID Date
      1 0 4 6
      1 8 5
      2 4 7
      3 3 3
      2 0 7 8
      1 5 9
      2 8 4


      I thought simply df.reset_index(level='Date', inplace=True, drop=True) would do it, but it does not.







      python pandas






      share|improve this question













      share|improve this question











      share|improve this question




      share|improve this question










      asked Aug 13 at 18:31









      KOB

      1,011617




      1,011617






















          4 Answers
          4






          active

          oldest

          votes

















          up vote
          4
          down vote



          accepted










          Using pd.MultiIndex.from_arrays and groupby + cumcount.



          df.index = pd.MultiIndex.from_arrays(
          [df.index.get_level_values(0), df.groupby(level=0).cumcount()],
          names=['ID', 'Date'])




          df
          col1 col2
          ID Date
          1 0 4 6
          1 8 5
          2 4 7
          3 3 3
          2 0 7 8
          1 5 9
          2 8 4


          This won't generalise to N levels, but there should be a df.index.set_levels equivalent I'm forgetting...






          share|improve this answer
















          • 2




            I find this solution easiest to follow.
            – KOB
            Aug 13 at 18:52

















          up vote
          4
          down vote













          You can groupby ID, then reset the index on each group using apply:



          new_df = (df.groupby(df.index.get_level_values('ID'))
          .apply(lambda x: x.reset_index()).drop(['ID','Date'],1))

          new_df.index = new_df.index.rename(['ID','Date'])

          >>> new_df
          col1 col2
          ID Date
          1 0 4 6
          1 8 5
          2 4 7
          3 3 3
          2 0 7 8
          1 5 9
          2 8 4





          share|improve this answer



























            up vote
            4
            down vote













            Using set_index and cumcount:



            tmp = df.reset_index('Date', drop=True)
            tmp.set_index(df.groupby(level=0).cumcount().rename('Date'), append=True)

            col1 col2
            ID Date
            1 0 4 6
            1 8 5
            2 4 7
            3 3 3
            2 0 7 8
            1 5 9
            2 8 4





            share|improve this answer





























              up vote
              3
              down vote













              New Answer



              Not as cool as the old answer but I'd rather be accurate than cool.



              from collections import defaultdict
              from itertools import count
              d = defaultdict(count)

              lbl =
              for a, *_ in df.index.values:
              lbl.append(next(d[a]))

              lvl = pd.RangeIndex(max(lbl) + 1)

              df.set_index(df.index.set_labels(lbl, 1).set_levels(lvl, 1))

              col1 col2
              ID Date
              1 0 4 6
              1 8 5
              2 4 7
              3 3 3
              2 0 7 8
              1 5 9
              2 8 4



              OLD ANSWER



              Do Not Use



              I misread the question. I didn't see that the new index needed to reset for every group.



              Hopefully useful to someone.



              You can use pandas.MultiIndex.set_levels



              n = 1
              lvl = df.index.levels[n]
              new_lvl = pd.RangeIndex(len(lvl))
              new_idx = df.index.set_levels(new_lvl, n)
              df.set_index(new_idx)

              col1 col2
              ID Date
              1 0 4 6
              1 8 5
              2 4 7
              3 3 3
              2 4 7 8
              5 5 9
              6 8 4



              One-line



              Yay! o/



              df.set_index(df.index.set_levels(pd.RangeIndex(len(df.index.levels[1])), 1))

              col1 col2
              ID Date
              1 0 4 6
              1 8 5
              2 4 7
              3 3 3
              2 4 7 8
              5 5 9
              6 8 4



              In place



              df.index.set_levels(pd.RangeIndex(len(df.index.levels[1])), 1, inplace=True)
              df

              col1 col2
              ID Date
              1 0 4 6
              1 8 5
              2 4 7
              3 3 3
              2 4 7 8
              5 5 9
              6 8 4





              share|improve this answer






















                Your Answer





                StackExchange.ifUsing("editor", function ()
                StackExchange.using("externalEditor", function ()
                StackExchange.using("snippets", function ()
                StackExchange.snippets.init();
                );
                );
                , "code-snippets");

                StackExchange.ready(function()
                var channelOptions =
                tags: "".split(" "),
                id: "1"
                ;
                initTagRenderer("".split(" "), "".split(" "), channelOptions);

                StackExchange.using("externalEditor", function()
                // Have to fire editor after snippets, if snippets enabled
                if (StackExchange.settings.snippets.snippetsEnabled)
                StackExchange.using("snippets", function()
                createEditor();
                );

                else
                createEditor();

                );

                function createEditor()
                StackExchange.prepareEditor(
                heartbeatType: 'answer',
                convertImagesToLinks: true,
                noModals: false,
                showLowRepImageUploadWarning: true,
                reputationToPostImages: 10,
                bindNavPrevention: true,
                postfix: "",
                onDemand: true,
                discardSelector: ".discard-answer"
                ,immediatelyShowMarkdownHelp:true
                );



                );













                 

                draft saved


                draft discarded


















                StackExchange.ready(
                function ()
                StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f51828518%2fpandas-reset-inner-level-of-multiindex%23new-answer', 'question_page');

                );

                Post as a guest






























                4 Answers
                4






                active

                oldest

                votes








                4 Answers
                4






                active

                oldest

                votes









                active

                oldest

                votes






                active

                oldest

                votes








                up vote
                4
                down vote



                accepted










                Using pd.MultiIndex.from_arrays and groupby + cumcount.



                df.index = pd.MultiIndex.from_arrays(
                [df.index.get_level_values(0), df.groupby(level=0).cumcount()],
                names=['ID', 'Date'])




                df
                col1 col2
                ID Date
                1 0 4 6
                1 8 5
                2 4 7
                3 3 3
                2 0 7 8
                1 5 9
                2 8 4


                This won't generalise to N levels, but there should be a df.index.set_levels equivalent I'm forgetting...






                share|improve this answer
















                • 2




                  I find this solution easiest to follow.
                  – KOB
                  Aug 13 at 18:52














                up vote
                4
                down vote



                accepted










                Using pd.MultiIndex.from_arrays and groupby + cumcount.



                df.index = pd.MultiIndex.from_arrays(
                [df.index.get_level_values(0), df.groupby(level=0).cumcount()],
                names=['ID', 'Date'])




                df
                col1 col2
                ID Date
                1 0 4 6
                1 8 5
                2 4 7
                3 3 3
                2 0 7 8
                1 5 9
                2 8 4


                This won't generalise to N levels, but there should be a df.index.set_levels equivalent I'm forgetting...






                share|improve this answer
















                • 2




                  I find this solution easiest to follow.
                  – KOB
                  Aug 13 at 18:52












                up vote
                4
                down vote



                accepted







                up vote
                4
                down vote



                accepted






                Using pd.MultiIndex.from_arrays and groupby + cumcount.



                df.index = pd.MultiIndex.from_arrays(
                [df.index.get_level_values(0), df.groupby(level=0).cumcount()],
                names=['ID', 'Date'])




                df
                col1 col2
                ID Date
                1 0 4 6
                1 8 5
                2 4 7
                3 3 3
                2 0 7 8
                1 5 9
                2 8 4


                This won't generalise to N levels, but there should be a df.index.set_levels equivalent I'm forgetting...






                share|improve this answer












                Using pd.MultiIndex.from_arrays and groupby + cumcount.



                df.index = pd.MultiIndex.from_arrays(
                [df.index.get_level_values(0), df.groupby(level=0).cumcount()],
                names=['ID', 'Date'])




                df
                col1 col2
                ID Date
                1 0 4 6
                1 8 5
                2 4 7
                3 3 3
                2 0 7 8
                1 5 9
                2 8 4


                This won't generalise to N levels, but there should be a df.index.set_levels equivalent I'm forgetting...







                share|improve this answer












                share|improve this answer



                share|improve this answer










                answered Aug 13 at 18:49









                coldspeed

                104k1789157




                104k1789157







                • 2




                  I find this solution easiest to follow.
                  – KOB
                  Aug 13 at 18:52












                • 2




                  I find this solution easiest to follow.
                  – KOB
                  Aug 13 at 18:52







                2




                2




                I find this solution easiest to follow.
                – KOB
                Aug 13 at 18:52




                I find this solution easiest to follow.
                – KOB
                Aug 13 at 18:52












                up vote
                4
                down vote













                You can groupby ID, then reset the index on each group using apply:



                new_df = (df.groupby(df.index.get_level_values('ID'))
                .apply(lambda x: x.reset_index()).drop(['ID','Date'],1))

                new_df.index = new_df.index.rename(['ID','Date'])

                >>> new_df
                col1 col2
                ID Date
                1 0 4 6
                1 8 5
                2 4 7
                3 3 3
                2 0 7 8
                1 5 9
                2 8 4





                share|improve this answer
























                  up vote
                  4
                  down vote













                  You can groupby ID, then reset the index on each group using apply:



                  new_df = (df.groupby(df.index.get_level_values('ID'))
                  .apply(lambda x: x.reset_index()).drop(['ID','Date'],1))

                  new_df.index = new_df.index.rename(['ID','Date'])

                  >>> new_df
                  col1 col2
                  ID Date
                  1 0 4 6
                  1 8 5
                  2 4 7
                  3 3 3
                  2 0 7 8
                  1 5 9
                  2 8 4





                  share|improve this answer






















                    up vote
                    4
                    down vote










                    up vote
                    4
                    down vote









                    You can groupby ID, then reset the index on each group using apply:



                    new_df = (df.groupby(df.index.get_level_values('ID'))
                    .apply(lambda x: x.reset_index()).drop(['ID','Date'],1))

                    new_df.index = new_df.index.rename(['ID','Date'])

                    >>> new_df
                    col1 col2
                    ID Date
                    1 0 4 6
                    1 8 5
                    2 4 7
                    3 3 3
                    2 0 7 8
                    1 5 9
                    2 8 4





                    share|improve this answer












                    You can groupby ID, then reset the index on each group using apply:



                    new_df = (df.groupby(df.index.get_level_values('ID'))
                    .apply(lambda x: x.reset_index()).drop(['ID','Date'],1))

                    new_df.index = new_df.index.rename(['ID','Date'])

                    >>> new_df
                    col1 col2
                    ID Date
                    1 0 4 6
                    1 8 5
                    2 4 7
                    3 3 3
                    2 0 7 8
                    1 5 9
                    2 8 4






                    share|improve this answer












                    share|improve this answer



                    share|improve this answer










                    answered Aug 13 at 18:37









                    sacul

                    18.4k31434




                    18.4k31434




















                        up vote
                        4
                        down vote













                        Using set_index and cumcount:



                        tmp = df.reset_index('Date', drop=True)
                        tmp.set_index(df.groupby(level=0).cumcount().rename('Date'), append=True)

                        col1 col2
                        ID Date
                        1 0 4 6
                        1 8 5
                        2 4 7
                        3 3 3
                        2 0 7 8
                        1 5 9
                        2 8 4





                        share|improve this answer


























                          up vote
                          4
                          down vote













                          Using set_index and cumcount:



                          tmp = df.reset_index('Date', drop=True)
                          tmp.set_index(df.groupby(level=0).cumcount().rename('Date'), append=True)

                          col1 col2
                          ID Date
                          1 0 4 6
                          1 8 5
                          2 4 7
                          3 3 3
                          2 0 7 8
                          1 5 9
                          2 8 4





                          share|improve this answer
























                            up vote
                            4
                            down vote










                            up vote
                            4
                            down vote









                            Using set_index and cumcount:



                            tmp = df.reset_index('Date', drop=True)
                            tmp.set_index(df.groupby(level=0).cumcount().rename('Date'), append=True)

                            col1 col2
                            ID Date
                            1 0 4 6
                            1 8 5
                            2 4 7
                            3 3 3
                            2 0 7 8
                            1 5 9
                            2 8 4





                            share|improve this answer














                            Using set_index and cumcount:



                            tmp = df.reset_index('Date', drop=True)
                            tmp.set_index(df.groupby(level=0).cumcount().rename('Date'), append=True)

                            col1 col2
                            ID Date
                            1 0 4 6
                            1 8 5
                            2 4 7
                            3 3 3
                            2 0 7 8
                            1 5 9
                            2 8 4






                            share|improve this answer














                            share|improve this answer



                            share|improve this answer








                            edited Aug 13 at 19:07

























                            answered Aug 13 at 18:48









                            user3483203

                            24.1k62147




                            24.1k62147




















                                up vote
                                3
                                down vote













                                New Answer



                                Not as cool as the old answer but I'd rather be accurate than cool.



                                from collections import defaultdict
                                from itertools import count
                                d = defaultdict(count)

                                lbl =
                                for a, *_ in df.index.values:
                                lbl.append(next(d[a]))

                                lvl = pd.RangeIndex(max(lbl) + 1)

                                df.set_index(df.index.set_labels(lbl, 1).set_levels(lvl, 1))

                                col1 col2
                                ID Date
                                1 0 4 6
                                1 8 5
                                2 4 7
                                3 3 3
                                2 0 7 8
                                1 5 9
                                2 8 4



                                OLD ANSWER



                                Do Not Use



                                I misread the question. I didn't see that the new index needed to reset for every group.



                                Hopefully useful to someone.



                                You can use pandas.MultiIndex.set_levels



                                n = 1
                                lvl = df.index.levels[n]
                                new_lvl = pd.RangeIndex(len(lvl))
                                new_idx = df.index.set_levels(new_lvl, n)
                                df.set_index(new_idx)

                                col1 col2
                                ID Date
                                1 0 4 6
                                1 8 5
                                2 4 7
                                3 3 3
                                2 4 7 8
                                5 5 9
                                6 8 4



                                One-line



                                Yay! o/



                                df.set_index(df.index.set_levels(pd.RangeIndex(len(df.index.levels[1])), 1))

                                col1 col2
                                ID Date
                                1 0 4 6
                                1 8 5
                                2 4 7
                                3 3 3
                                2 4 7 8
                                5 5 9
                                6 8 4



                                In place



                                df.index.set_levels(pd.RangeIndex(len(df.index.levels[1])), 1, inplace=True)
                                df

                                col1 col2
                                ID Date
                                1 0 4 6
                                1 8 5
                                2 4 7
                                3 3 3
                                2 4 7 8
                                5 5 9
                                6 8 4





                                share|improve this answer


























                                  up vote
                                  3
                                  down vote













                                  New Answer



                                  Not as cool as the old answer but I'd rather be accurate than cool.



                                  from collections import defaultdict
                                  from itertools import count
                                  d = defaultdict(count)

                                  lbl =
                                  for a, *_ in df.index.values:
                                  lbl.append(next(d[a]))

                                  lvl = pd.RangeIndex(max(lbl) + 1)

                                  df.set_index(df.index.set_labels(lbl, 1).set_levels(lvl, 1))

                                  col1 col2
                                  ID Date
                                  1 0 4 6
                                  1 8 5
                                  2 4 7
                                  3 3 3
                                  2 0 7 8
                                  1 5 9
                                  2 8 4



                                  OLD ANSWER



                                  Do Not Use



                                  I misread the question. I didn't see that the new index needed to reset for every group.



                                  Hopefully useful to someone.



                                  You can use pandas.MultiIndex.set_levels



                                  n = 1
                                  lvl = df.index.levels[n]
                                  new_lvl = pd.RangeIndex(len(lvl))
                                  new_idx = df.index.set_levels(new_lvl, n)
                                  df.set_index(new_idx)

                                  col1 col2
                                  ID Date
                                  1 0 4 6
                                  1 8 5
                                  2 4 7
                                  3 3 3
                                  2 4 7 8
                                  5 5 9
                                  6 8 4



                                  One-line



                                  Yay! o/



                                  df.set_index(df.index.set_levels(pd.RangeIndex(len(df.index.levels[1])), 1))

                                  col1 col2
                                  ID Date
                                  1 0 4 6
                                  1 8 5
                                  2 4 7
                                  3 3 3
                                  2 4 7 8
                                  5 5 9
                                  6 8 4



                                  In place



                                  df.index.set_levels(pd.RangeIndex(len(df.index.levels[1])), 1, inplace=True)
                                  df

                                  col1 col2
                                  ID Date
                                  1 0 4 6
                                  1 8 5
                                  2 4 7
                                  3 3 3
                                  2 4 7 8
                                  5 5 9
                                  6 8 4





                                  share|improve this answer
























                                    up vote
                                    3
                                    down vote










                                    up vote
                                    3
                                    down vote









                                    New Answer



                                    Not as cool as the old answer but I'd rather be accurate than cool.



                                    from collections import defaultdict
                                    from itertools import count
                                    d = defaultdict(count)

                                    lbl =
                                    for a, *_ in df.index.values:
                                    lbl.append(next(d[a]))

                                    lvl = pd.RangeIndex(max(lbl) + 1)

                                    df.set_index(df.index.set_labels(lbl, 1).set_levels(lvl, 1))

                                    col1 col2
                                    ID Date
                                    1 0 4 6
                                    1 8 5
                                    2 4 7
                                    3 3 3
                                    2 0 7 8
                                    1 5 9
                                    2 8 4



                                    OLD ANSWER



                                    Do Not Use



                                    I misread the question. I didn't see that the new index needed to reset for every group.



                                    Hopefully useful to someone.



                                    You can use pandas.MultiIndex.set_levels



                                    n = 1
                                    lvl = df.index.levels[n]
                                    new_lvl = pd.RangeIndex(len(lvl))
                                    new_idx = df.index.set_levels(new_lvl, n)
                                    df.set_index(new_idx)

                                    col1 col2
                                    ID Date
                                    1 0 4 6
                                    1 8 5
                                    2 4 7
                                    3 3 3
                                    2 4 7 8
                                    5 5 9
                                    6 8 4



                                    One-line



                                    Yay! o/



                                    df.set_index(df.index.set_levels(pd.RangeIndex(len(df.index.levels[1])), 1))

                                    col1 col2
                                    ID Date
                                    1 0 4 6
                                    1 8 5
                                    2 4 7
                                    3 3 3
                                    2 4 7 8
                                    5 5 9
                                    6 8 4



                                    In place



                                    df.index.set_levels(pd.RangeIndex(len(df.index.levels[1])), 1, inplace=True)
                                    df

                                    col1 col2
                                    ID Date
                                    1 0 4 6
                                    1 8 5
                                    2 4 7
                                    3 3 3
                                    2 4 7 8
                                    5 5 9
                                    6 8 4





                                    share|improve this answer














                                    New Answer



                                    Not as cool as the old answer but I'd rather be accurate than cool.



                                    from collections import defaultdict
                                    from itertools import count
                                    d = defaultdict(count)

                                    lbl =
                                    for a, *_ in df.index.values:
                                    lbl.append(next(d[a]))

                                    lvl = pd.RangeIndex(max(lbl) + 1)

                                    df.set_index(df.index.set_labels(lbl, 1).set_levels(lvl, 1))

                                    col1 col2
                                    ID Date
                                    1 0 4 6
                                    1 8 5
                                    2 4 7
                                    3 3 3
                                    2 0 7 8
                                    1 5 9
                                    2 8 4



                                    OLD ANSWER



                                    Do Not Use



                                    I misread the question. I didn't see that the new index needed to reset for every group.



                                    Hopefully useful to someone.



                                    You can use pandas.MultiIndex.set_levels



                                    n = 1
                                    lvl = df.index.levels[n]
                                    new_lvl = pd.RangeIndex(len(lvl))
                                    new_idx = df.index.set_levels(new_lvl, n)
                                    df.set_index(new_idx)

                                    col1 col2
                                    ID Date
                                    1 0 4 6
                                    1 8 5
                                    2 4 7
                                    3 3 3
                                    2 4 7 8
                                    5 5 9
                                    6 8 4



                                    One-line



                                    Yay! o/



                                    df.set_index(df.index.set_levels(pd.RangeIndex(len(df.index.levels[1])), 1))

                                    col1 col2
                                    ID Date
                                    1 0 4 6
                                    1 8 5
                                    2 4 7
                                    3 3 3
                                    2 4 7 8
                                    5 5 9
                                    6 8 4



                                    In place



                                    df.index.set_levels(pd.RangeIndex(len(df.index.levels[1])), 1, inplace=True)
                                    df

                                    col1 col2
                                    ID Date
                                    1 0 4 6
                                    1 8 5
                                    2 4 7
                                    3 3 3
                                    2 4 7 8
                                    5 5 9
                                    6 8 4






                                    share|improve this answer














                                    share|improve this answer



                                    share|improve this answer








                                    edited Aug 13 at 19:36

























                                    answered Aug 13 at 19:05









                                    piRSquared

                                    140k19118248




                                    140k19118248



























                                         

                                        draft saved


                                        draft discarded















































                                         


                                        draft saved


                                        draft discarded














                                        StackExchange.ready(
                                        function ()
                                        StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f51828518%2fpandas-reset-inner-level-of-multiindex%23new-answer', 'question_page');

                                        );

                                        Post as a guest













































































                                        Popular posts from this blog

                                        How to check contact read email or not when send email to Individual?

                                        Christian Cage

                                        How to properly install USB display driver for Fresco Logic FL2000DX on Ubuntu?