mjhjmtu

Need to convert selected Attributes from the block of text between blank lines in the LDIF(text) file and convert it to CSV file with comma separated delimiter, similar to below example:

Example:

LDIF file (as input):

<Blank Line>
AA: User11_Value1
BB: User11_Value2
CC: User11_Value3
DD: User11 Space Value4
<Blank Line>
AA: User22_Value1
BB: User22_Value2
CC: User22_Value3
DD: User22 Space Value4
<Blank Line>

Convert it to CSV format (as output):

AA,BB,DD
User11_Value1,User11_Value2,User11 Space Value4
User22_Value1,User22_Value2,User22 Space Value4

This is the script which reads LDIF from STDIN and output as CSV

#!/bin/bash

#

# Converts LDIF data to CSV.

# Doesn't handle comments very well. Use -LLL with ldapsearch to remove them.

#

# 2010-03-07

# dsimmons@squiz.co.uk

#


# Show usage if we don't have the right params

if [ "$1" == "" ]; then

 echo ""

 echo "Usage: cat ldif.txt | $0 <attributes> [...]"

 echo "Where <attributes> contains a list of space-separated attributes to include in the CSV. LDIF data is read from stdin."

 echo ""

 exit 99

fi


ATTRS="$*"


c=0

while read line; do


 # Skip LDIF comments

 [ "$line:0:1" == "#" ] && continue;


 # If this line is blank then it's the end of this record, and the beginning

 # of a new one.

 #

 if [ "$line" == "" ]; then


 output=""


 # Output the CSV record

 for i in $ATTRS; do


 eval data=$RECORD_$c_$i

 output=$output"$data",


 unset RECORD_$c_$i


 done


 # Remove trailing ',' and echo the output

 output=$output%,

 echo $output


 # Increase the counter

 c=$(($c+1))

 fi


 # Separate attribute name/value at the semicolon (LDIF format)

 attr=$line%%:*

 value=$line#*: 


 # Save all the attributes in variables for now (ie. buffer), because the data

 # isn't necessarily in a set order.

 #

 for i in $ATTRS; do

 if [ "$attr" == "$i" ]; then

 eval RECORD_$c_$attr="$value"

 fi

 done


done

Click here for more

I see some serious deficiencies in simple scripts like this:

• no proper handling of base64-encoded data which is used for non-ASCII chars or octet strings


• no proper handling of line-wrapping


• LDAP data model has multi-valued attributes

If you don't want to fix this yourself after reading through RFC 2849 I'd recommend to implement a short Python script using the python-ldap sub-module ldif and built-in csv module.

with Miller (http://johnkerl.org/miller/doc) and sed is very short and easy

sed 's/://g' input.txt | mlr --x2c cut -x -f CC

gives you

AA,BB,DD
User11_Value1,User11_Value2,User11 Space Value4
User22_Value1,User22_Value2,User22 Space Value4

Whit sed I remove the : to obtain one of the native Miller input format (XTAB), than I convert XTAB to CSV with --x2c and at the end I remove CC field with cut.