Final answer:
The given expression simplifies to roughly 2n log n + n³ log n for large n, and the n³ log n term dominates the growth. Thus, the correct representation of g(n) is n² log n, which is option d.
Step-by-step explanation:
The expression given is n log (n² + 1) + n² log n. To analyze this expression and determine its order of growth, we use the properties of logarithms and the definition of big O notation.
The logarithm of a number raised to an exponent is the product of the exponent and the logarithm of the number, which simplifies our second term to n³ log n. The first term can be approximated for large values of n since n² + 1 behaves like n² as n grows large, resulting in n log n², which is 2n log n.
Adding both simplified terms, we get approximately 2n log n + n³ log n. This expression is dominated by the n³ log n term as n tends to infinity. Therefore, we can ignore the linear logarithmic term in comparison to the cubic logarithmic term.
The function g(n) that represents the growth of the given expression is n³ log n, which corresponds to option d.n² log n.