If A and B are two square matrices such that $B={{A}^{-1}}BA$ , then ${{\left( A+B \right)}^{2}}$ is equal to.
(a) ${{A}^{2}}+{{B}^{2}}$
(b) $O$
(c) ${{A}^{2}}+2AB+{{B}^{2}}$
(d) $A+B$

Hint: For solving this question first we will pre multiply by the matrix $A$ in the equation $B={{A}^{-1}}BA$ and prove that in this problem $B\cdot A=A\cdot B$ . After that, we will multiply $\left( A+B \right)$ with itself to get ${{\left( A+B \right)}^{2}}$ and solve further of the correct answer.

Complete step-by-step solution -
Given:
It is given that, $A$ and $B$ are two square matrices such that $B={{A}^{-1}}BA$ and we have to solve for the result of ${{\left( A+B \right)}^{2}}$ .
Now, we have the following equation:
$B={{A}^{-1}}\cdot B\cdot A$
Now, pre-multiply by the matrix $A$ in the above equation. Then,
$\begin{align}
  & B={{A}^{-1}}\cdot B\cdot A \\
 & \Rightarrow A\cdot B=A\cdot {{A}^{-1}}\cdot B\cdot A \\
\end{align}$
Now, we know that $A\cdot {{A}^{-1}}=I$ where $I$ is the identity matrix of the same order. Then,
$\begin{align}
  & A\cdot B=A\cdot {{A}^{-1}}\cdot B\cdot A \\
 & \Rightarrow A\cdot B=I\cdot B\cdot A \\
\end{align}$
Now, we know that when we multiply an identity matrix with any other matrix $B$, then we will get the resultant matrix as $B$ itself. Then,
$\begin{align}
  & A\cdot B=I\cdot B\cdot A \\
 & \Rightarrow A\cdot B=B\cdot A...................\left( 1 \right) \\
\end{align}$
Now, we will multiply $\left( A+B \right)$ with itself to get ${{\left( A+B \right)}^{2}}$ . Then,
$\begin{align}
  & {{\left( A+B \right)}^{2}}=\left( A+B \right)\times \left( A+B \right) \\
 & \Rightarrow {{\left( A+B \right)}^{2}}=A\cdot A+A\cdot B+B\cdot A+B\cdot B \\
 & \Rightarrow {{\left( A+B \right)}^{2}}={{A}^{2}}+A\cdot B+B\cdot A+{{B}^{2}} \\
\end{align}$
Now, from equation (1) we can write $B\cdot A=A\cdot B$ in the above equation. Then,
$\begin{align}
  & {{\left( A+B \right)}^{2}}={{A}^{2}}+A\cdot B+B\cdot A+{{B}^{2}} \\
 & \Rightarrow {{\left( A+B \right)}^{2}}={{A}^{2}}+A\cdot B+A\cdot B+{{B}^{2}} \\
 & \Rightarrow {{\left( A+B \right)}^{2}}={{A}^{2}}+2A\cdot B+{{B}^{2}} \\
\end{align}$
Now, from the above result, we conclude that, if $B={{A}^{-1}}BA$ then, ${{\left( A+B \right)}^{2}}={{A}^{2}}+2A\cdot B+{{B}^{2}}$ .
Hence, option (c) will be the correct option.

Note: Here, the student should first understand what is asked in the problem and then proceed in the right direction to get the correct answer quickly. And we should proceed stepwise while solving for smooth calculation. Moreover, though the question is very easy, here $B\cdot A=A\cdot B$ but first we should prove it and then solve further as per the rules of the matrix algebra. And don’t remember $B\cdot A=A\cdot B$ as a formula in general for other questions.