Steve Ray has a list of 35 loaded Questions for "Bible Christians" (quotation marks his)(link to the whole list). This is number 14/35. I'm trying to provide the answers in a common format, for easy reference.
14) Most of the books of the New Testament were written to address very specific problems in the early Church, and none of them are a systematic presentation of Christian faith and theology. On what biblical basis do Protestants think that everything that the apostles taught is captured in the New Testament writings?
We don't necessarily think that everything that the apostles taught is captured in the New Testament writings.
1) But everything that we know the apostles taught, we know from Scripture.
2) The apostles, being human beings, may have sometimes taught something that was an error. They were not guaranteed always to be infallible whenever they taught.
3) Furthermore, the gospels do present the Christian faith in an orderly way (though not arranged like a textbook).
4) And the point of the gospels being written is told to us, and it is so that we might believe and be saved:
John 20:31 But these are written, that ye might believe that Jesus is the Christ, the Son of God; and that believing ye might have life through his name.
5) The characterization "[m]ost of the books of the New Testament were written to address very specific problems in the early Church," is very misleading at best. While the books do sometimes correct specific problems (sometimes very specific problems), virtually the entire body of Christian writings from the earliest generations through the Reformation era understood that point of Scripture was not simply to correct specific problems at the time.
6) Indeed, Scripture itself explains:
2 Timothy 3:16-17
All scripture is given by inspiration of God, and is profitable for doctrine, for reproof, for correction, for instruction in righteousness: that the man of God may be perfect, throughly furnished unto all good works.