关于中性化实现的疑问

[absatu]

请教一下，关于中性化实现，你设计了17种方法，应该是覆盖了从线性模型到机器学习模型的场景。

疑问1：中性化一般用哑变量回归来做吧？为什么会有17种完全不同的实现呢？能否简单讲讲？
疑问2：针对LGB模型，你建议用哪种中性化方法比较合适？
疑问3：怎么校验中性化后的数据准确性？

对于疑问3，我对比了3个方案，均没有得到完全相同的中性化后数据，甚至量纲级别都不同

1. 聚宽平台提供的中性化函数

代码：

from jqdata import *
import pandas as pd
import numpy as np
from jqfactor import neutralize

def initialize(context):
    # 生成数据
    data = pd.DataFrame({
    '600036.XSHG': [1.5, 801780, 8111.3009],   # 招商银行，2026-03-31 close:39.32，流通8111.3009亿, sw_l1: 801780
    '600519.XSHG': [2.3, 801120,18157.9181],   # 茅台，2026-03-31 close:1450，流通18157.9181亿, sw_l1: 801120
    '300059.XSHE': [0.8, 801790, 2519.5017]    # 东方财富，2026-03-31 close:18.89，流通2519.5017亿, sw_l1: 801790
        }, index=['feature_a', 'industry', 'circulating_market_cap'])

    print(f"before neutralize: \n{data}")
    
    # 取 feature_a
    feature_df = data.loc[['feature_a', 'industry', 'circulating_market_cap']]
    
    # neutralize 只传 feature_a（DataFrame），axis=1
    # how 直接传列名列表（中性化因子），不要传 dict
    df = neutralize(
        feature_df.loc[['feature_a']],       # 只传 feature_a
        how=['sw_l1', 'circulating_market_cap'],
        date='2026-03-31',
        axis=1
    )
                            
    print(f"after neutralize: \n{df}")

日志：

2019-01-01 00:00:00 - INFO  - before neutralize: 
                        600036.XSHG  600519.XSHG  300059.XSHE
feature_a                    1.5000       2.3000       0.8000
industry                801780.0000  801120.0000  801790.0000
circulating_market_cap    8111.3009   18157.9181    2519.5017

2019-01-01 00:00:00 - INFO  - after neutralize: 
            600036.XSHG   600519.XSHG   300059.XSHE
feature_a  8.881784e-16  1.776357e-15  1.110223e-16

2. 你提供的中性化函数

代码：

def compare():
    data = pd.DataFrame(
        [
            {
                "datetime": "2026-03-31",
                "symbol": "600036.XSHG",
                "close": 39.32,
                "circulating_market_cap": 8111.3009,
                "feature_a": 1.5,
                "industry": 801780,
            },
            {
                "datetime": "2026-03-31",
                "symbol": "600519.XSHG",
                "close": 1450,
                "circulating_market_cap": 18157.9181,
                "feature_a": 2.3,
                "industry": 801120,
            },
            {
                "datetime": "2026-03-31",
                "symbol": "300059.XSHE",
                "close": 18.89,
                "circulating_market_cap": 2519.5017,
                "feature_a": 0.8,
                "industry": 801790,
            },
        ]
    )

    purifier = AlphaPurifier(base_df=data, factor_name="feature_a", trade_date_col="datetime", symbol_col="symbol")

    # 默认 multiOLS，按行业和市值中性化
    method = ["multiOLS", "lasso", "ridge", "elasticnet", "polynomial", "kernelridge", "huber", "rank", "theilsen", "randomforest", "GBDT", "ICA", "PCA", "bayesianridge", "partialcorrelation"]
    for m in method:
        try:
            result = purifier.neutralize(m, ["industry", "circulating_market_cap"]).to_result()
            print(f"{m}: \n{result}") 
        except Exception as e:
            print(f"{m} failed with error: {e}")

日志（其中有一些方法调用报错）：

multiOLS: 
     datetime       symbol    close  circulating_market_cap     feature_a  industry
0  2026-03-31  600036.XSHG    39.32               8111.3009  2.273737e-13    801780
1  2026-03-31  600519.XSHG  1450.00              18157.9181  2.726708e-13    801120
2  2026-03-31  300059.XSHE    18.89               2519.5017  2.728928e-13    801790
lasso: 
     datetime       symbol    close  circulating_market_cap     feature_a  industry
0  2026-03-31  600036.XSHG    39.32               8111.3009 -3.027208e-14    801780
1  2026-03-31  600519.XSHG  1450.00              18157.9181  1.502502e-14    801120
2  2026-03-31  300059.XSHE    18.89               2519.5017  1.524706e-14    801790
ridge: 
     datetime       symbol    close  circulating_market_cap     feature_a  industry
0  2026-03-31  600036.XSHG    39.32               8111.3009 -8.637504e-19    801780
1  2026-03-31  600519.XSHG  1450.00              18157.9181  3.083064e-19    801120
2  2026-03-31  300059.XSHE    18.89               2519.5017  5.554439e-19    801790
elasticnet: 
     datetime       symbol    close  circulating_market_cap     feature_a  industry
0  2026-03-31  600036.XSHG    39.32               8111.3009 -8.637504e-19    801780
1  2026-03-31  600519.XSHG  1450.00              18157.9181  3.083064e-19    801120
2  2026-03-31  300059.XSHE    18.89               2519.5017  5.554439e-19    801790
polynomial: 
     datetime       symbol    close  circulating_market_cap     feature_a  industry
0  2026-03-31  600036.XSHG    39.32               8111.3009 -8.185202e-33    801780
1  2026-03-31  600519.XSHG  1450.00              18157.9181  7.703720e-34    801120
2  2026-03-31  300059.XSHE    18.89               2519.5017  4.911121e-33    801790
5.20s - Error patching args (debugger not attached to subprocess).
Traceback (most recent call last):
  File "/Users/abab/.vscode/extensions/ms-python.debugpy-2025.18.0-darwin-x64/bundled/libs/debugpy/_vendored/pydevd/_pydev_bundle/pydev_monkey.py", line 541, in patch_args
    new_args.append(_get_python_c_args(host, port, code, unquoted_args, SetupHolder.setup))
                    ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/abab/.vscode/extensions/ms-python.debugpy-2025.18.0-darwin-x64/bundled/libs/debugpy/_vendored/pydevd/_pydev_bundle/pydev_monkey.py", line 193, in _get_python_c_args
    if "__future__" in code:
       ^^^^^^^^^^^^^^^^^^^^
TypeError: a bytes-like object is required, not 'str'
No module named joblib.externals.loky.backend.popen_loky_posix
No module named joblib.externals.loky.backend.popen_loky_posix
No module named joblib.externals.loky.backend.popen_loky_posix
No module named joblib.externals.loky.backend.popen_loky_posix
No module named joblib.externals.loky.backend.popen_loky_posix
No module named joblib.externals.loky.backend.popen_loky_posix
No module named joblib.externals.loky.backend.popen_loky_posix
No module named joblib.externals.loky.backend.popen_loky_posix
kernelridge failed with error: A worker process managed by the executor was unexpectedly terminated. This could be caused by a segmentation fault while calling the function or by an excessive memory usage causing the Operating System to kill the worker.

The exit codes of the workers are {EXIT(0)}
Detailed tracebacks of the workers should have been printed to stderr in the executor process if faulthandler was not disabled.
No module named joblib.externals.loky.backend.popen_loky_posix
No module named joblib.externals.loky.backend.popen_loky_posix
No module named joblib.externals.loky.backend.popen_loky_posix
No module named joblib.externals.loky.backend.popen_loky_posix
No module named joblib.externals.loky.backend.popen_loky_posix
No module named joblib.externals.loky.backend.popen_loky_posix
No module named joblib.externals.loky.backend.popen_loky_posix
No module named joblib.externals.loky.backend.popen_loky_posix
huber failed with error: A worker process managed by the executor was unexpectedly terminated. This could be caused by a segmentation fault while calling the function or by an excessive memory usage causing the Operating System to kill the worker.

The exit codes of the workers are {EXIT(0)}
Detailed tracebacks of the workers should have been printed to stderr in the executor process if faulthandler was not disabled.
No module named joblib.externals.loky.backend.popen_loky_posix
No module named joblib.externals.loky.backend.popen_loky_posix
No module named joblib.externals.loky.backend.popen_loky_posix
No module named joblib.externals.loky.backend.popen_loky_posix
No module named joblib.externals.loky.backend.popen_loky_posix
No module named joblib.externals.loky.backend.popen_loky_posix
No module named joblib.externals.loky.backend.popen_loky_posix
No module named joblib.externals.loky.backend.popen_loky_posix
rank failed with error: A worker process managed by the executor was unexpectedly terminated. This could be caused by a segmentation fault while calling the function or by an excessive memory usage causing the Operating System to kill the worker.

The exit codes of the workers are {EXIT(0)}
Detailed tracebacks of the workers should have been printed to stderr in the executor process if faulthandler was not disabled.
No module named joblib.externals.loky.backend.popen_loky_posix
No module named joblib.externals.loky.backend.popen_loky_posix
No module named joblib.externals.loky.backend.popen_loky_posix
No module named joblib.externals.loky.backend.popen_loky_posix
No module named joblib.externals.loky.backend.popen_loky_posix
No module named joblib.externals.loky.backend.popen_loky_posix
No module named joblib.externals.loky.backend.popen_loky_posix
No module named joblib.externals.loky.backend.popen_loky_posix
theilsen failed with error: A worker process managed by the executor was unexpectedly terminated. This could be caused by a segmentation fault while calling the function or by an excessive memory usage causing the Operating System to kill the worker.

The exit codes of the workers are {EXIT(0)}
Detailed tracebacks of the workers should have been printed to stderr in the executor process if faulthandler was not disabled.
randomforest: 
     datetime       symbol    close  circulating_market_cap     feature_a  industry
0  2026-03-31  600036.XSHG    39.32               8111.3009 -7.203620e-33    801780
1  2026-03-31  600519.XSHG  1450.00              18157.9181  1.751954e-33    801120
2  2026-03-31  300059.XSHE    18.89               2519.5017  5.892704e-33    801790
GBDT: 
     datetime       symbol    close  circulating_market_cap     feature_a  industry
0  2026-03-31  600036.XSHG    39.32               8111.3009 -7.350633e-33    801780
1  2026-03-31  600519.XSHG  1450.00              18157.9181  1.604942e-33    801120
2  2026-03-31  300059.XSHE    18.89               2519.5017  5.745691e-33    801790
No module named joblib.externals.loky.backend.popen_loky_posix
No module named joblib.externals.loky.backend.popen_loky_posix
No module named joblib.externals.loky.backend.popen_loky_posix
No module named joblib.externals.loky.backend.popen_loky_posix
No module named joblib.externals.loky.backend.popen_loky_posix
No module named joblib.externals.loky.backend.popen_loky_posix
No module named joblib.externals.loky.backend.popen_loky_posix
No module named joblib.externals.loky.backend.popen_loky_posix
ICA failed with error: A worker process managed by the executor was unexpectedly terminated. This could be caused by a segmentation fault while calling the function or by an excessive memory usage causing the Operating System to kill the worker.

The exit codes of the workers are {EXIT(0)}
Detailed tracebacks of the workers should have been printed to stderr in the executor process if faulthandler was not disabled.
PCA: 
     datetime       symbol    close  circulating_market_cap     feature_a  industry
0  2026-03-31  600036.XSHG    39.32               -0.187392 -2.736911e-48  0.564279
1  2026-03-31  600519.XSHG  1450.00                1.080440 -2.394797e-48 -1.154603
2  2026-03-31  300059.XSHE    18.89               -0.893048  0.000000e+00  0.590323
bayesianridge: 
     datetime       symbol    close  circulating_market_cap     feature_a  industry
0  2026-03-31  600036.XSHG    39.32               -0.187392 -9.502278e-49  0.564279
1  2026-03-31  600519.XSHG  1450.00                1.080440  1.207075e-49 -1.154603
2  2026-03-31  300059.XSHE    18.89               -0.893048  8.295202e-49  0.590323
partialcorrelation: 
     datetime       symbol    close  circulating_market_cap     feature_a  industry
0  2026-03-31  600036.XSHG    39.32               -0.187392 -2.556966e-61  0.564279
1  2026-03-31  600519.XSHG  1450.00                1.080440 -2.600266e-61 -1.154603
2  2026-03-31  300059.XSHE    18.89               -0.893048 -2.578237e-61  0.590323

3.我自己写的版本

代码（基于Qlib):

class CSNeutralize:
    def __init__(
        self,
        factor_group,  # list of column tuples to neutralize
        risk_cols=None,  # list of column tuples as risk factors
        industry_col=None,  # single column tuple for industry
        ridge_alpha=1e-6,
    ):
        self.factor_group = factor_group
        self.risk_cols = risk_cols or []
        self.industry_col = industry_col
        self.ridge_alpha = ridge_alpha
        self._fit_done = False

    def fit(self, df: pd.DataFrame):
        # factor_cols：排除风险因子和行业列
        self.factor_cols = [c for c in self.factor_group if c not in self.risk_cols and c != self.industry_col]
        self._fit_done = True
        return self

    def __call__(self, data: pd.DataFrame):
        if not self._fit_done:
            self.fit(data)

        df_fg = data[self.factor_group].copy()
        Y_all = df_fg[self.factor_cols].values.astype(float)

        # 风险因子
        if self.risk_cols:
            risk_all = df_fg[self.risk_cols].values.astype(float)
        else:
            risk_all = np.empty((len(df_fg), 0))

        # 行业
        if self.industry_col:
            industry = df_fg[self.industry_col].astype(str).str.slice(0, 4).values
            industry_codes, industry_idx = np.unique(industry, return_inverse=True)
            n_ind = len(industry_codes) - 1
        else:
            industry_idx = None
            n_ind = 0

        # 日期
        dates = df_fg.index.get_level_values("datetime").values
        unique_dates, date_idx = np.unique(dates, return_inverse=True)
        n_dates = len(unique_dates)

        for d in range(n_dates):
            if d % 50 == 0:
                logger.info(f"Neutralize progress: {d}/{n_dates} dates")

            mask = date_idx == d
            Y = Y_all[mask]
            X_parts = []

            # intercept
            X_parts.append(np.ones((Y.shape[0], 1)))

            # numeric risk
            if self.risk_cols:
                X_parts.append(risk_all[mask])

            # industry dummy
            if self.industry_col:
                ind = industry_idx[mask]
                dummy = np.zeros((len(ind), n_ind))
                valid = ind > 0
                dummy[np.arange(len(ind))[valid], ind[valid] - 1] = 1
                X_parts.append(dummy)

            # 合并设计矩阵
            X = np.hstack(X_parts)

            # 过滤无效行
            valid_mask = np.isfinite(X).all(axis=1) & np.isfinite(Y).all(axis=1)
            if valid_mask.sum() == 0:
                continue

            Xv = np.ascontiguousarray(X[valid_mask])
            Yv = np.ascontiguousarray(Y[valid_mask])

            # 一次性对多列中性化
            resid = self._lstsq_resid(Xv, Yv, self.ridge_alpha)
            Y[valid_mask, :] = resid
            Y_all[mask, :] = Y

        logger.info("Neutralize finished")
        data.loc[:, self.factor_cols] = Y_all.astype(float)
        return data

    @staticmethod
    @njit(parallel=True, fastmath=True)
    def _lstsq_resid(X, Y, ridge_alpha):
        n_samples, n_features = X.shape
        n_targets = Y.shape[1]
        resid = np.empty_like(Y)
        XtX = X.T @ X + np.eye(n_features) * ridge_alpha
        beta = np.linalg.solve(XtX, X.T @ Y)
        for i in prange(n_samples):
            for j in range(n_targets):
                resid[i, j] = Y[i, j] - X[i, :] @ beta[:, j]
        return resid
if __name__ == "__main__":
    cols = pd.MultiIndex.from_tuples(
        [
            ("feature", "feature_a"),
            ("feature", "industry"),
            ("feature", "log_mktcap"),
        ]
    )
    index = pd.MultiIndex.from_arrays(
        [
            pd.to_datetime(["2026-03-31"] * 3),
            ["600036.XSHG", "600519.XSHG", "300059.XSHE"],
        ],
        names=["datetime", "instrument"],
    )
    data = pd.DataFrame(
        [
            [1.5, 801780, 8111.3009],
            [2.3, 801120, 18157.9181],
            [0.8, 801790, 2519.5017],
        ],
        index=index,
        columns=cols,
    )
    print(f"Original data: \n{data}")

    processor = CSNeutralize(
        factor_group=[
            ("feature", "feature_a"),
            ("feature", "industry"),
            ("feature", "log_mktcap"),
        ],
        # factor_group=[("feature", "feature_a"), ("feature", "feature_b"),
        #               ("feature", "industry"), ("feature", "log_mktcap")],
        risk_cols=[("feature", "log_mktcap")],
        industry_col=("feature", "industry"),
    )
    processor.fit(data)
    neutralized_data = processor(data)
    print(f"Neutralized data: \n{neutralized_data}")

日志：

Original data: 
                         feature                     
                       feature_a industry  log_mktcap
datetime   instrument                                
2026-03-31 600036.XSHG       1.5   801780   8111.3009
           600519.XSHG       2.3   801120  18157.9181
           300059.XSHE       0.8   801790   2519.5017
Neutralized data: 
                             feature                     
                           feature_a industry  log_mktcap
datetime   instrument                                    
2026-03-31 600036.XSHG  1.192460e-06   801780   8111.3009
           600519.XSHG -4.307257e-07   801120  18157.9181
           300059.XSHE -7.347961e-07   801790   2519.5017

[eliasswu]

你好，要调用AlphaPurier.get_methods()功能查看参数，哑变量不是直接放在第一个list的，

result = purifier.neutralize(m, ["industry", "circulating_market_cap"])这里参数位置不对，要写在第二个位置，或者用dummy_cols＝list指定

此外，哑变量去做中性化只是一种方式，还有很多其他的直接中性化，你list的这个地方，应该是填其他因子值的，不是填哑变量的，假如说要剔除动量，beta这些因子的相关性，他们的频率是和行情数据要一致的，把其他因子col名的list放在第一个位置，第二个参数位置才是做哑变量处理的

提到LGB，可以说到更清楚一点，具体是做什么？中性化的准确性要了解源码才可以更好做到了，因为每个quant的理念不一样，但是应该不会差别太大

函数具体用法你可以看看examples文件里面AlphaPurifier的示例，谢谢你～

Hello～Please call the  AlphaPurifier.get_methods()  function to check the parameters. Dummy variables are not placed directly in the first list.

In the code below:

python

result = purifier.neutralize(m, ["industry", "circulating_market_cap"])
 

the parameter positions are incorrect. You should place them in the second position, or specify them via the  dummy_cols=list  argument.

Furthermore, dummy variables are not mandatory for neutralization.
The list in the first parameter is intended for names of other factor columns, not dummy variables. For instance, if you want to remove correlation from factors such as momentum and beta (which share the same data frequency as your market data), you should pass the column names of these factors as a list into the first parameter. The second parameter is reserved for dummy variable processing.

Please refer to the AlphaPurifier examples in the  examples  file for reference. Thank you～