mjhjmtu

I would like to define a command transp having eventually one argument : the name of the matrix and finally two outputs.

1. 
   transpA is the matrix A^T between parenthesis,


2. 
   transp A is just the matrix A^T.

I tried this command :

newcommandtransp[1]
ifstrempty#1^texttbfT ^texttbfT left( #1 right)

but to print the transpose symbol I have to write transp. Can I modify the previous command in order to just write transp (as mentioned in 2.) ?

According to the standard TeX syntax, transpA and transp A are completely equivalent.

You might do in the following way:

documentclassarticle
usepackageamsmath

makeatletter
DeclareRobustCommandtransp%
 @ifnextcharbgrouptransp@parentransp@simple

newcommandtransp@paren[1](#1)^T
newcommandtransp@simple[1]#1^T
makeatother

begindocument

$transp A+transpB+C$

enddocument

but I would avoid it, because it's confusing.

I find the following much better. You explicitly mark where you want parentheses by adding *.

documentclassarticle
usepackageamsmath
usepackagexparse

NewDocumentCommandtranspsm%
 IfBooleanTF#1(#2)^T#2^T%


begindocument

$transpA+transp*B+C$

enddocument

The following seems to work, but I doubt it is a good idea in general. Usually foo A and foo A give the same result for macros with one argument and the braces are needed in case the argument consists of more than one token. Indeed I would say that it is good practice to use braces for mandatory arguments even if they enclose only one token.

Note that transp without braces can only accept one token as its argument, so transp A+B is transp A and +B. In particular then, transp mathbfA dies horribly.

documentclassarticle
usepackageamsmath

makeatletter
newcommand*transp@nb[1]#1^T
newcommand*transp@br[1](#1)^T
newcommandtransp
protecteddeftransp%
 @ifnextcharbgroup
 transp@br
 transp@nb
makeatother

begindocument
 beginalign*
 transp A \
 transpA
 endalign*
enddocument

A starred variant would be more common (see also egreg's answer)

documentclassarticle
usepackageamsmath

makeatletter
newcommand*transp@nb[1]#1^T
newcommand*transp@br[1](#1)^T
newcommandtransp
protecteddeftransp%
 @ifstar
 transp@br
 transp@nb
makeatother

begindocument
 beginalign*
 transpA \
 transp*A
 endalign*
enddocument

but you could also use an optional argument (p for parentheses, b for brackets)

documentclassarticle
usepackageamsmath

makeatletter
newcommandtransp[2]%
 if#1p
 (#2)
 else
 if#1b
 [A]
 else
 A
 fi
 fi^T

makeatother

begindocument
 beginalign*
 transpA \
 transp[b]A
 endalign*
enddocument