[Spring] WebClient DataBufferLimitException WebFlux 오류 해결

글 작성시의 버전

Springboot : 2.3 버전 / Spring webflux : 5.2 버전

 

문제

WebClient를 통해 데이터를 가져오는데 데이터의 용량이 너무 커서 메모리 버퍼사이즈가 초과해서

DataBufferLimitException: Exceeded limit on max bytes to buffer : 262144

에러가 발생했습니다.

 

해결

application memory 이슈를 피하기 위해 Codecs는 버퍼 데이터 사이즈를 제한을 두었습니다. 

기본값으로 256KB이고, 만약 이 값이 충분하지 않을 경우에는 기본 코덱 설정값을 바꿔줘야합니다.

WebClient를 build시 codec 설정을 넣어주면 되는데요. 

 

특정사이즈(예 10MB)로 하고 싶다면? 10 * 1024 * 1024 

버퍼사이즈를 무한대로 하고 싶다고 하면 : -1  로 설정해주면 됩니다.

ExchangeStrategies exchangeStrategies = ExchangeStrategies.builder()
            .codecs(configurer -> configurer.defaultCodecs()
            .maxInMemorySize(10 * 1024 * 1024) //10MB로 설정
            //.maxInMemorySize(-1)) //제한없음
            .build(); 

this.webClient = webClientBuilder.exchangeStrategies(exchangeStrategies);

 

Spring에서 AbstractDataBufferDecoder 구현을 살펴보면 기본값인 256KB로 세팅이 되어있는걸 알 수 있습니다.

/**
 * Abstract base class for {@code Decoder} implementations that can decode
 * a {@code DataBuffer} directly to the target element type.
 *
 * <p>Sub-classes must implement {@link #decodeDataBuffer} to provide a way to
 * transform a {@code DataBuffer} to the target data type. The default
 * {@link #decode} implementation transforms each individual data buffer while
 * {@link #decodeToMono} applies "reduce" and transforms the aggregated buffer.
 *
 * <p>Sub-classes can override {@link #decode} in order to split the input stream
 * along different boundaries (e.g. on new line characters for {@code String})
 * or always reduce to a single data buffer (e.g. {@code Resource}).
 *
 * @author Rossen Stoyanchev
 * @since 5.0
 * @param <T> the element type
 */
@SuppressWarnings("deprecation")
public abstract class AbstractDataBufferDecoder<T> extends AbstractDecoder<T> {

	private int maxInMemorySize = 256 * 1024;


	protected AbstractDataBufferDecoder(MimeType... supportedMimeTypes) {
		super(supportedMimeTypes);
	}


	/**
	 * Configure a limit on the number of bytes that can be buffered whenever
	 * the input stream needs to be aggregated. This can be a result of
	 * decoding to a single {@code DataBuffer},
	 * {@link java.nio.ByteBuffer ByteBuffer}, {@code byte[]},
	 * {@link org.springframework.core.io.Resource Resource}, {@code String}, etc.
	 * It can also occur when splitting the input stream, e.g. delimited text,
	 * in which case the limit applies to data buffered between delimiters.
	 * <p>By default this is set to 256K.
	 * @param byteCount the max number of bytes to buffer, or -1 for unlimited
	 * @since 5.1.11
	 */
	public void setMaxInMemorySize(int byteCount) {
		this.maxInMemorySize = byteCount;
	}

https://github.com/spring-projects/spring-framework/blob/main/spring-core/src/main/java/org/springframework/core/codec/AbstractDataBufferDecoder.java

GitHub - spring-projects/spring-framework: Spring Framework

 

그 외. SpringFramework - MaxInMemorySize 설정

https://github.com/spring-projects/spring-framework/blob/main/src/docs/asciidoc/web/webflux-webclient.adoc#maxinmemorysize

 

 

참고로, Spring 버전에 따라 이전 버전에서는 코드에서가 아닌 application.yml 파일에서 property 값을 수정해주면 됩니다.

spring:
  codec:
    max-in-memory-size: 10MB