Merge two DataFrames based on columns and values of a specific column with Pandas in Python 3.x

Multi tool use
Multi tool use

The name of the pictureThe name of the pictureThe name of the pictureClash Royale CLAN TAG#URR8PPP











up vote
6
down vote

favorite
1












Hello i have a problem which i am not able to implement a solution on.
I have following two DataFrames:



>>> df1
A B date
1 1 01-2016
2 1 02-2017
1 2 03-2017
2 2 04-2020

>>> df2
A B 01-2016 02-2017 03-2017 04.2020
1 1 0.10 0.22 0.55 0.77
2 1 0.20 0.12 0.99 0.125
1 2 0.13 0.15 0.15 0.245
2 2 0.33 0.1 0.888 0.64


What i want is following DataFrame:



>>> df3
A B date value
1 1 01-2016 0.10
2 1 02-2017 0.12
1 2 03-2017 0.15
2 2 04-2020 0.64


I already tried following:



 summarize_dates = self.summarize_specific_column(data=df1, column='date')

 for date in summarize_dates:
 left_on = np.append(left_on, date)
 right_on = np.append(right_on, merge_columns.upper())
 result = pd.merge(left=df2, right=df1,
 left_on=left_on, right_on=right_on,
 how='right')
 print(result)


This does not work. Can you help me and suggest a more comfortable implementation? Manyy thanks in advance!






python pandas csv merge







share|improve this question



















  • 1




    The easiest way to do this would be to melt df2 and then do a left join on df1.
    – CJ59
    3 hours ago






  • 1




    Thanks for your answer. But what do you mean with melt the Frame? Edit: Was already answered. :)
    – Michael Gann
    3 hours ago















up vote
6
down vote

favorite
1












Hello i have a problem which i am not able to implement a solution on.
I have following two DataFrames:



>>> df1
A B date
1 1 01-2016
2 1 02-2017
1 2 03-2017
2 2 04-2020

>>> df2
A B 01-2016 02-2017 03-2017 04.2020
1 1 0.10 0.22 0.55 0.77
2 1 0.20 0.12 0.99 0.125
1 2 0.13 0.15 0.15 0.245
2 2 0.33 0.1 0.888 0.64


What i want is following DataFrame:



>>> df3
A B date value
1 1 01-2016 0.10
2 1 02-2017 0.12
1 2 03-2017 0.15
2 2 04-2020 0.64


I already tried following:



 summarize_dates = self.summarize_specific_column(data=df1, column='date')

 for date in summarize_dates:
 left_on = np.append(left_on, date)
 right_on = np.append(right_on, merge_columns.upper())
 result = pd.merge(left=df2, right=df1,
 left_on=left_on, right_on=right_on,
 how='right')
 print(result)


This does not work. Can you help me and suggest a more comfortable implementation? Manyy thanks in advance!






python pandas csv merge







share|improve this question



















  • 1




    The easiest way to do this would be to melt df2 and then do a left join on df1.
    – CJ59
    3 hours ago






  • 1




    Thanks for your answer. But what do you mean with melt the Frame? Edit: Was already answered. :)
    – Michael Gann
    3 hours ago













up vote
6
down vote

favorite
1









up vote
6
down vote

favorite
1






1





Hello i have a problem which i am not able to implement a solution on.
I have following two DataFrames:



>>> df1
A B date
1 1 01-2016
2 1 02-2017
1 2 03-2017
2 2 04-2020

>>> df2
A B 01-2016 02-2017 03-2017 04.2020
1 1 0.10 0.22 0.55 0.77
2 1 0.20 0.12 0.99 0.125
1 2 0.13 0.15 0.15 0.245
2 2 0.33 0.1 0.888 0.64


What i want is following DataFrame:



>>> df3
A B date value
1 1 01-2016 0.10
2 1 02-2017 0.12
1 2 03-2017 0.15
2 2 04-2020 0.64


I already tried following:



 summarize_dates = self.summarize_specific_column(data=df1, column='date')

 for date in summarize_dates:
 left_on = np.append(left_on, date)
 right_on = np.append(right_on, merge_columns.upper())
 result = pd.merge(left=df2, right=df1,
 left_on=left_on, right_on=right_on,
 how='right')
 print(result)


This does not work. Can you help me and suggest a more comfortable implementation? Manyy thanks in advance!






python pandas csv merge







share|improve this question















Hello i have a problem which i am not able to implement a solution on.
I have following two DataFrames:



>>> df1
A B date
1 1 01-2016
2 1 02-2017
1 2 03-2017
2 2 04-2020

>>> df2
A B 01-2016 02-2017 03-2017 04.2020
1 1 0.10 0.22 0.55 0.77
2 1 0.20 0.12 0.99 0.125
1 2 0.13 0.15 0.15 0.245
2 2 0.33 0.1 0.888 0.64


What i want is following DataFrame:



>>> df3
A B date value
1 1 01-2016 0.10
2 1 02-2017 0.12
1 2 03-2017 0.15
2 2 04-2020 0.64


I already tried following:



 summarize_dates = self.summarize_specific_column(data=df1, column='date')

 for date in summarize_dates:
 left_on = np.append(left_on, date)
 right_on = np.append(right_on, merge_columns.upper())
 result = pd.merge(left=df2, right=df1,
 left_on=left_on, right_on=right_on,
 how='right')
 print(result)


This does not work. Can you help me and suggest a more comfortable implementation? Manyy thanks in advance!






python pandas csv merge




python pandas csv merge






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited 3 hours ago

























asked 3 hours ago









Michael Gann

343




343







  • 1




    The easiest way to do this would be to melt df2 and then do a left join on df1.
    – CJ59
    3 hours ago






  • 1




    Thanks for your answer. But what do you mean with melt the Frame? Edit: Was already answered. :)
    – Michael Gann
    3 hours ago













  • 1




    The easiest way to do this would be to melt df2 and then do a left join on df1.
    – CJ59
    3 hours ago






  • 1




    Thanks for your answer. But what do you mean with melt the Frame? Edit: Was already answered. :)
    – Michael Gann
    3 hours ago








1




1




The easiest way to do this would be to melt df2 and then do a left join on df1.
– CJ59
3 hours ago




The easiest way to do this would be to melt df2 and then do a left join on df1.
– CJ59
3 hours ago




1




1




Thanks for your answer. But what do you mean with melt the Frame? Edit: Was already answered. :)
– Michael Gann
3 hours ago





Thanks for your answer. But what do you mean with melt the Frame? Edit: Was already answered. :)
– Michael Gann
3 hours ago













2 Answers
2






active

oldest

votes

















up vote
9
down vote













You can melt df2 and then merge using the default 'inner' merge



df3 = df1.merge(df2.melt(id_vars = ['A', 'B'], var_name='date'))

 A B date value
0 1 1 01-2016 0.10
1 2 1 02-2017 0.12
2 1 2 03-2017 0.15
3 2 2 04-2020 0.64





share|improve this answer


















  • 1




    Thank you i will try that out!
    – Michael Gann
    3 hours ago






  • 1




    Works fine! Thank you very much!
    – Michael Gann
    2 hours ago

















up vote
3
down vote













Using lookup 



df1['value']=df2.set_index(['A','B']).lookup(df1.set_index(['A','B']).index,df1.date)
df1
Out[228]: 
 A B date value
0 1 1 01-2016 0.10
1 2 1 02-2017 0.12
2 1 2 03-2017 0.15
3 2 2 04-2020 0.64





share|improve this answer




















  • Works fine! Thank you very much!
    – Michael Gann
    2 hours ago










Your Answer






StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");

StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);

else
createEditor();

);

function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);



);













 

draft saved


draft discarded


















StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53214900%2fmerge-two-dataframes-based-on-columns-and-values-of-a-specific-column-with-panda%23new-answer', 'question_page');

);

Post as a guest

















2 Answers
2






active

oldest

votes








2 Answers
2






active

oldest

votes









active

oldest

votes






active

oldest

votes








up vote
9
down vote













You can melt df2 and then merge using the default 'inner' merge



df3 = df1.merge(df2.melt(id_vars = ['A', 'B'], var_name='date'))

 A B date value
0 1 1 01-2016 0.10
1 2 1 02-2017 0.12
2 1 2 03-2017 0.15
3 2 2 04-2020 0.64





share|improve this answer


















  • 1




    Thank you i will try that out!
    – Michael Gann
    3 hours ago






  • 1




    Works fine! Thank you very much!
    – Michael Gann
    2 hours ago














up vote
9
down vote













You can melt df2 and then merge using the default 'inner' merge



df3 = df1.merge(df2.melt(id_vars = ['A', 'B'], var_name='date'))

 A B date value
0 1 1 01-2016 0.10
1 2 1 02-2017 0.12
2 1 2 03-2017 0.15
3 2 2 04-2020 0.64





share|improve this answer


















  • 1




    Thank you i will try that out!
    – Michael Gann
    3 hours ago






  • 1




    Works fine! Thank you very much!
    – Michael Gann
    2 hours ago












up vote
9
down vote










up vote
9
down vote









You can melt df2 and then merge using the default 'inner' merge



df3 = df1.merge(df2.melt(id_vars = ['A', 'B'], var_name='date'))

 A B date value
0 1 1 01-2016 0.10
1 2 1 02-2017 0.12
2 1 2 03-2017 0.15
3 2 2 04-2020 0.64





share|improve this answer














You can melt df2 and then merge using the default 'inner' merge



df3 = df1.merge(df2.melt(id_vars = ['A', 'B'], var_name='date'))

 A B date value
0 1 1 01-2016 0.10
1 2 1 02-2017 0.12
2 1 2 03-2017 0.15
3 2 2 04-2020 0.64






share|improve this answer














share|improve this answer



share|improve this answer








edited 3 hours ago

























answered 3 hours ago









Vaishali

16.2k3927




16.2k3927







  • 1




    Thank you i will try that out!
    – Michael Gann
    3 hours ago






  • 1




    Works fine! Thank you very much!
    – Michael Gann
    2 hours ago












  • 1




    Thank you i will try that out!
    – Michael Gann
    3 hours ago






  • 1




    Works fine! Thank you very much!
    – Michael Gann
    2 hours ago







1




1




Thank you i will try that out!
– Michael Gann
3 hours ago




Thank you i will try that out!
– Michael Gann
3 hours ago




1




1




Works fine! Thank you very much!
– Michael Gann
2 hours ago




Works fine! Thank you very much!
– Michael Gann
2 hours ago












up vote
3
down vote













Using lookup 



df1['value']=df2.set_index(['A','B']).lookup(df1.set_index(['A','B']).index,df1.date)
df1
Out[228]: 
 A B date value
0 1 1 01-2016 0.10
1 2 1 02-2017 0.12
2 1 2 03-2017 0.15
3 2 2 04-2020 0.64





share|improve this answer




















  • Works fine! Thank you very much!
    – Michael Gann
    2 hours ago














up vote
3
down vote













Using lookup 



df1['value']=df2.set_index(['A','B']).lookup(df1.set_index(['A','B']).index,df1.date)
df1
Out[228]: 
 A B date value
0 1 1 01-2016 0.10
1 2 1 02-2017 0.12
2 1 2 03-2017 0.15
3 2 2 04-2020 0.64





share|improve this answer




















  • Works fine! Thank you very much!
    – Michael Gann
    2 hours ago












up vote
3
down vote










up vote
3
down vote









Using lookup 



df1['value']=df2.set_index(['A','B']).lookup(df1.set_index(['A','B']).index,df1.date)
df1
Out[228]: 
 A B date value
0 1 1 01-2016 0.10
1 2 1 02-2017 0.12
2 1 2 03-2017 0.15
3 2 2 04-2020 0.64





share|improve this answer












Using lookup 



df1['value']=df2.set_index(['A','B']).lookup(df1.set_index(['A','B']).index,df1.date)
df1
Out[228]: 
 A B date value
0 1 1 01-2016 0.10
1 2 1 02-2017 0.12
2 1 2 03-2017 0.15
3 2 2 04-2020 0.64






share|improve this answer












share|improve this answer



share|improve this answer










answered 3 hours ago









W-B

90.5k72754




90.5k72754











  • Works fine! Thank you very much!
    – Michael Gann
    2 hours ago
















  • Works fine! Thank you very much!
    – Michael Gann
    2 hours ago















Works fine! Thank you very much!
– Michael Gann
2 hours ago




Works fine! Thank you very much!
– Michael Gann
2 hours ago

















 

draft saved


draft discarded















































 


draft saved


draft discarded














StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53214900%2fmerge-two-dataframes-based-on-columns-and-values-of-a-specific-column-with-panda%23new-answer', 'question_page');

);

Post as a guest
































































FBOlOQTFyKxZa Z7,0AhA,KrmQcY bsyTsV,p FX NLqf5 UxkCCp4L 9n,4qt
-
4LnE,tv 5fLKrva

Popular posts from this blog

How to check contact read email or not when send email to Individual?

Image
Clash Royale CLAN TAG #URR8PPP up vote 1 down vote favorite I'm using WordPress 4.9.8, CiviCRM to 5.5.1, I usually send email to contact by Search> Find contacts View contact details Action> Send email Send email ok, Contact received mail ok like picture But status only Email sent though contact read email or not. So, can CiviCRM can change status to Email read when contact read email? wordpress email share | improve this question asked Sep 26 at 0:12 ToanLuong 49 9 add a comment  |  up vote 1 down vote favorite I'm using WordPress 4.9.8, CiviCRM to 5.5.1, I usually send email to contact by Search> Find contacts View contact details Action> Send email Send email ok, Contact received mail ok like picture But status only Email sent though contact read email or not. So, can CiviCRM can change status to Email read when contact read email? wordpress email share | improve this questi...
Read more

How many registers does an x86_64 CPU actually have?

Image
Clash Royale CLAN TAG #URR8PPP up vote 2 down vote favorite I am currently learning reverse engineering and am studying the flags register. I had in my mind that rflags was just another name for one of the 16 general purpose registers, for example rax or rbx . But it looks like rflags is actually an additional register. So that makes 17 registers in total... how many more could there be? I have spent at least an hour on this and found numerous different answers. The best answer so far is this, which says that there are 40 registers in total. 16 General Purpose Registers 2 Status Registers 6 Code Segment Registers 16 SSE Registers 8 FPU/MMX Registers But if I add that up, I get 48. Could anybody provide an official answer on how many registers an x86_64 CPU has (e.g. an Intel i7). Additionally, I have seen references to 'hardware' and 'architectural' registers. What are those registers and how many are there? register x86-64 share | improve this...
Read more

Displaying single band from multi-band raster using QGIS

Image
Clash Royale CLAN TAG #URR8PPP 1 How can I extract a single band from multi-band raster in QGIS? I have an remote sensed image which has 6 bands (including NDVI band), I want to display each band separately, but have no idea how to do. I have seen some questions similar here but none worked for me. The original image (has 6 bands) is: I want to display the band 6 which should be like this: But I tried gdal_translate, and couldn't get the correct result. What I have got is: qgis raster multi-band share | improve this question edited Mar 5 at 0:53 Summer asked Mar 4 at 6:42 Summer Summer 23 6 Is this any help gis.stackexchange.com/questions/220658/… ? if not gis.stackexchange.com/questions/62133/… might help. – Michael Stimson Mar 4 at 6:46 Thanks for answering but when I used gdal_translate, qgis showed that 'Error 4: Kayena.tif: No such file or directory". Would you know how to fi...
Read more