Carbon nanotubes (CNTs) have been regarded as an ideal reinforcements of high-performance composites with enormous applications. In this paper, the effects of pinhole defect are investigated for carbon nanotube based nanocomposites using a 3D representative volume element (RVE) with long CNTs. The CNT is modeled as a continuum hollow cylindrical shape elastic material with pinholes in it. These defects are considered on the single wall (CNTs). The mechanical properties such as Young’s modulus of elasticity are evaluated for various pinhole locations and number of defects. The influence of the pinhole defects on the nanocomposite is studied under an axial load condition. Numerical equations are used to extract the effective material properties for the different geometries of RVEs with nondefective CNTs. The field-emission microscopy (FEM) results obtained for nondefective CNTs are consistent with the analytical results for cylindrical RVEs, which validate the proposed model. It is observed that the presence of pinhole defects significantly reduces the effective reinforcement when compared with nondefective nanotubes, and this reinforcement decreases with the increase in the number of pinhole defects. It is also found from the simulation results that the geometry of RVE does not have much significance on the stiffness of nanocomposites.