Network Intrusion Detection Systems (NIDSs) have emerged as powerful tools for detecting and preventing malicious attacks over both the Internet and Intranet. String matching, which is one of the most important functions of NIDS, demands exceptionally high performance to match the content of network traffic against a predefined database of malicious patterns. Much work has been done in this field; however, they result in low memory efficiency. Due to the available on-chip memory and the number of I/O pins of Field Programmable Gate Arrays (FPGAs), state-of-the-art designs cannot support large dictionaries without using high-latency external DRAM. We propose a novel Memory efficient Architecture for large-scale String Matching, namely MASM, based on pipelined binary search tree. Our design provides a high-throughput matching module, which can be used as the building block to process arbitrary-length patterns. With memory efficiency close to $1$ byte/char, MASM can support a dictionary of over $4$ MB (regardless of the size of the alphabet), using a single state-of-the-art FPGA device. This efficiency is comparable to that of a Ternary Content Addressable Memory (TCAM)-based solution. The architecture can also be easily partitioned, so as to use external SRAM to handle even larger dictionaries of over $8$ MB. Our implementation results show a sustained throughput of $3.2$ Gbps, even when external SRAM is used. The MASM module can be simply duplicated to accept multiple characters per cycle, leading to scalable throughput with respect to the number of characters processed in each cycle. Dictionary update involves only rewriting the memory content, which can be done quickly without reconfiguring the chip.