使用卷积神经网络实现图片去摩尔纹_数据_卷积_图片

来源 | OSCHINA 社区

作者 | 华为云开发者 李长安

摘要：本项目主要介绍了如何使用卷积神经网络去检测翻拍图片，主要为摩尔纹图片；其主要创新点在于网络结构上，将图片的高低频信息分开处理。

本文分享自华为云社区《图片去摩尔纹简述与代码实现》，作者： 李长安。

1 前言

当感光元件像素的空间频率与影像中条纹的空间频率接近时，可能产生一种新的波浪形的干扰图案，即所谓的摩尔纹。传感器的网格状纹理构成了一个这样的图案。当图案中的细条状结构与传感器的结构以小角度交叉时，这种效应也会在图像中产生明显的干扰。这种现象在一些细密纹理情况下，比如时尚摄影中的布料上，非常普遍。这种摩尔纹可能通过亮度也可能通过颜色来展现。但是在这里，仅针对在翻拍过程中产生的图像摩尔纹进行处理。

翻拍即从计算机屏幕上捕获图片，或对着屏幕拍摄图片；该方式会在图片上产生摩尔纹现象

论文主要处理思路

对原图作 Haar 变换得到四个下采样特征图（原图下二采样 cA、Horizontal 横向高频 cH、Vertical 纵向高频 cV、Diagonal 斜向高频 cD）

然后分别利用四个独立的 CNN 对四个下采样特征图卷积池化，提取特征信息

原文随后对三个高频信息卷积池化后的结果的每个 channel、每个像素点比对，取 max

将上一步得到的结果和 cA 卷积池化后的结果作笛卡尔积

2、网络结构复现

如下图所示，本项目复现了论文的图像去摩尔纹方法，并对数据处理部分进行了修改，并且网络结构上也参考了源码中的结构，对图片产生四个下采样特征图，而不是论文中的三个，具体处理方式大家可以参考一下网络结构。

展开全文

importmath

importpaddle

importpaddle.nnasnn

importpaddle.nn.functionalasF

# import pywt

frompaddle.nnimportLinear,Dropout,ReLU

frompaddle.nnimportConv2D,MaxPool2D

classmcnn(nn.Layer):

def__init__(self,num_classes=1000):

super(mcnn,self).__init__

self.num_classes=num_classes

self._conv1_LL=Conv2D(3,32,7,stride=2,padding=1,)

# self.bn1_LL = nn.BatchNorm2D(128)

self._conv1_LH=Conv2D(3,32,7,stride=2,padding=1,)

# self.bn1_LH = nn.BatchNorm2D(256)

self._conv1_HL=Conv2D(3,32,7,stride=2,padding=1,)

# self.bn1_HL = nn.BatchNorm2D(512)

self._conv1_HH=Conv2D(3,32,7,stride=2,padding=1,)

# self.bn1_HH = nn.BatchNorm2D(256)

self.pool_1_LL=nn.MaxPool2D(kernel_size=2,stride=2,padding=0)

self.pool_1_LH=nn.MaxPool2D(kernel_size=2,stride=2,padding=0)

self.pool_1_HL=nn.MaxPool2D(kernel_size=2,stride=2,padding=0)

self.pool_1_HH=nn.MaxPool2D(kernel_size=2,stride=2,padding=0)

self._conv2=Conv2D(32,16,3,stride=2,padding=1,)

self.pool_2=nn.MaxPool2D(kernel_size=2,stride=2,padding=0)

self.dropout2=Dropout(p=0.5)

self._conv3=Conv2D(16,32,3,stride=2,padding=1,)

self.pool_3=nn.MaxPool2D(kernel_size=2,stride=2,padding=0)

self._conv4=Conv2D(32,32,3,stride=2,padding=1,)

self.pool_4=nn.MaxPool2D(kernel_size=2,stride=2,padding=0)

self.dropout4=Dropout(p=0.5)

# self.bn1_HH = nn.BatchNorm1D(256)

self._fc1=Linear(in_features=64,out_features=num_classes)

self.dropout5=Dropout(p=0.5)

self._fc2=Linear(in_features=2,out_features=num_classes)

defforward(self,inputs1,inputs2,inputs3,inputs4):

x1_LL=self._conv1_LL(inputs1)

x1_LL=F.relu(x1_LL)

x1_LH=self._conv1_LH(inputs2)

x1_LH=F.relu(x1_LH)

x1_HL=self._conv1_HL(inputs3)

x1_HL=F.relu(x1_HL)

x1_HH=self._conv1_HH(inputs4)

x1_HH=F.relu(x1_HH)

pool_x1_LL=self.pool_1_LL(x1_LL)

pool_x1_LH=self.pool_1_LH(x1_LH)

pool_x1_HL=self.pool_1_HL(x1_HL)

pool_x1_HH=self.pool_1_HH(x1_HH)

temp=paddle.maximum(pool_x1_LH,pool_x1_HL)

avg_LH_HL_HH=paddle.maximum(temp,pool_x1_HH)

inp_merged=paddle.multiply(pool_x1_LL,avg_LH_HL_HH)

x2=self._conv2(inp_merged)

x2=F.relu(x2)

x2=self.pool_2(x2)

x2=self.dropout2(x2)

x3=self._conv3(x2)

x3=F.relu(x3)

x3=self.pool_3(x3)

x4=self._conv4(x3)

x4=F.relu(x4)

x4=self.pool_4(x4)

x4=self.dropout4(x4)

x4=paddle.flatten(x4,start_axis=1,stop_axis=-1)

x5=self._fc1(x4)

x5=self.dropout5(x5)

out=self._fc2(x5)

returnout

model_res=mcnn(num_classes=2)

paddle.summary(model_res,[(1,3,512,384),(1,3,512,384),(1,3,512,384),(1,3,512,384)])

Layer(type)InputShapeOutputShapeParam#

Conv2D-1[[1,3,512,384]][1,32,254,190]4,736

Conv2D-2[[1,3,512,384]][1,32,254,190]4,736

Conv2D-3[[1,3,512,384]][1,32,254,190]4,736

Conv2D-4[[1,3,512,384]][1,32,254,190]4,736

MaxPool2D-1[[1,32,254,190]][1,32,127,95]0

MaxPool2D-2[[1,32,254,190]][1,32,127,95]0

MaxPool2D-3[[1,32,254,190]][1,32,127,95]0

MaxPool2D-4[[1,32,254,190]][1,32,127,95]0

Conv2D-5[[1,32,127,95]][1,16,64,48]4,624

MaxPool2D-5[[1,16,64,48]][1,16,32,24]0

Dropout-1[[1,16,32,24]][1,16,32,24]0

Conv2D-6[[1,16,32,24]][1,32,16,12]4,640

MaxPool2D-6[[1,32,16,12]][1,32,8,6]0

Conv2D-7[[1,32,8,6]][1,32,4,3]9,248

MaxPool2D-7[[1,32,4,3]][1,32,2,1]0

Dropout-2[[1,32,2,1]][1,32,2,1]0

Linear-1[[1,64]][1,2]130

Dropout-3[[1,2]][1,2]0

Linear-2[[1,2]][1,2]6

Total params:37,592

Trainable params:37,592

Non-trainable params:0

Inputsize(MB):9.00

Forward/backwardpasssize(MB):59.54

Paramssize(MB):0.14

EstimatedTotalSize(MB):68.68

{'total_params':37592,'trainable_params':37592}3、数据预处理

与源代码不同的是，本项目将图像的小波分解部分集成在了数据读取部分，即改为了线上进行小波分解，而不是源代码中的线下进行小波分解并且保存图片。首先，定义小波分解的函数

!pip install PyWavelets

importnumpy asnp

importpywt

defsplitFreqBands(img, levRows, levCols):

halfRow = int(levRows/ 2)

halfCol = int(levCols/ 2)

LL = img[ 0:halfRow, 0:halfCol]

LH = img[ 0:halfRow, halfCol:levCols]

HL = img[halfRow:levRows, 0:halfCol]

HH = img[halfRow:levRows, halfCol:levCols]

returnLL, LH, HL, HH

defhaarDWT1D(data, length):

avg0 = 0.5;

avg1 = 0.5;

dif0 = 0.5;

dif1 = -0.5;

temp = np.empty_like(data)

# temp = temp.astype(float)

temp = temp.astype(np.uint8)

h = int(length/ 2)

fori inrange(h):

k = i* 2

temp[i] = data[k] * avg0 + data[k + 1] * avg1;

temp[i + h] = data[k] * dif0 + data[k + 1] * dif1;

data[:] = temp

# computes the homography coefficients for PIL.Image.transform using point correspondences

deffwdHaarDWT2D(img):

img = np.array(img)

levRows = img.shape[ 0];

levCols = img.shape[ 1];

# img = img.astype(float)

img = img.astype(np.uint8)

fori inrange(levRows):

row = img[i,:]

haarDWT1D(row, levCols)

img[i,:] = row

forj inrange(levCols):

col = img[:,j]

haarDWT1D(col, levRows)

img[:,j] = col

returnsplitFreqBands(img, levRows, levCols)

!cd "data/data188843/"&& unzip -q 'total_images.zip'

importos

recapture_keys = [ 'ValidationMoire']

original_keys = [ 'ValidationClear']

defget_image_label_from_folder_name(folder_name):

:param folder_name:

:return:

forkey inoriginal_keys:

ifkey infolder_name:

return'original'

forkey inrecapture_keys:

ifkey infolder_name:

return'recapture'

return'unclear'

label_name2label_id = {

'original': 0,

'recapture': 1,}

src_image_dir = "data/data188843/total_images"

dst_file = "data/data188843/total_images/train.txt"

image_folder = [file forfile inos.listdir(src_image_dir)]

print(image_folder)

image_anno_list = []

forfolder inimage_folder:

label_name = get_image_label_from_folder_name(folder)

# label_id = label_name2label_id.get(label_name, 0)

label_id = label_name2label_id[label_name]

folder_path = os.path.join(src_image_dir, folder)

image_file_list = [file forfile inos.listdir(folder_path) if

file.endswith( '.jpg') orfile.endswith( '.jpeg') or

file.endswith( '.JPG') orfile.endswith( '.JPEG') orfile.endswith( '.png')]

forimage_file inimage_file_list:

# if need_root_dir:

# image_path = os.path.join(folder_path, image_file)

# else:

image_path = image_file

image_anno_list.append(folder + "/"+image_path + "\t"+ str(label_id) + '\n')

dst_path = os.path.dirname(src_image_dir)

ifnotos.path.exists(dst_path):

os.makedirs(dst_path)

withopen(dst_file, 'w') asfd:

fd.writelines(image_anno_list)

importpaddle

importnumpy asnp

importpandas aspd

importPIL.Image asImage

frompaddle.vision importtransforms

# from haar2D import fwdHaarDWT2D

paddle.disable_static

# 定义数据预处理

data_transforms = transforms.Compose([

transforms.Resize(size=( 448, 448)),

transforms.ToTensor, # transpose操作 + (img / 255)

# transforms.Normalize( # 减均值 除标准差

# mean=[0.31169346, 0.25506335, 0.12432463],

# std=[0.34042713, 0.29819837, 0.1375536])

#计算过程：output[channel] = (input[channel] - mean[channel]) / std[channel]

# 构建Dataset

classMyDataset(paddle.io.Dataset):

步骤一：继承paddle.io.Dataset类

def__init__(self, train_img_list, val_img_list, train_label_list, val_label_list, mode= 'train', ) :

步骤二：实现构造函数，定义数据读取方式，划分训练和测试数据集

super(MyDataset, self).__init__

self.img = []

self.label = []

# 借助pandas读csv的库

self.train_images = train_img_list

self.test_images = val_img_list

self.train_label = train_label_list

self.test_label = val_label_list

ifmode == 'train':

# 读train_images的数据

forimg,la inzip(self.train_images, self.train_label):

self.img.append( '/home/aistudio/data/data188843/total_images/'+img)

self.label.append(paddle.to_tensor(int(la), dtype= 'int64'))

else:

# 读test_images的数据

forimg,la inzip(self.test_images, self.test_label):

self.img.append( '/home/aistudio/data/data188843/total_images/'+img)

self.label.append(paddle.to_tensor(int(la), dtype= 'int64'))

defload_img(self, image_path):

# 实际使用时使用Pillow相关库进行图片读取即可，这里我们对数据先做个模拟

image = Image.open(image_path).convert( 'RGB')

# image = data_transforms(image)

returnimage

def__getitem__(self, index):

步骤三：实现__getitem__方法，定义指定index时如何获取数据，并返回单条数据（训练数据，对应的标签）

image = self.load_img(self.img[index])

LL, LH, HL, HH = fwdHaarDWT2D(image)

label = self.label[index]

# print(LL.shape)

# print(LH.shape)

# print(HL.shape)

# print(HH.shape)

LL = data_transforms(LL)

LH = data_transforms(LH)

HL = data_transforms(HL)

HH = data_transforms(HH)

print(type(LL))

print(LL.dtype)

returnLL, LH, HL, HH, np.array(label, dtype= 'int64')

def__len__(self):

步骤四：实现__len__方法，返回数据集总数目

returnlen(self.img)

image_file_txt = '/home/aistudio/data/data188843/total_images/train.txt'

withopen(image_file_txt) asfd:

lines = fd.readlines

train_img_list = list

train_label_list = list

forline inlines:

split_list = line.strip.split

image_name, label_id = split_list

train_img_list.append(image_name)

train_label_list.append(label_id)

# print(train_img_list)

# print(train_label_list)

# 测试定义的数据集

train_dataset = MyDataset(mode= 'train',train_label_list=train_label_list, train_img_list=train_img_list, val_img_list=train_img_list, val_label_list=train_label_list)

# test_dataset = MyDataset(mode='test')

# 构建训练集数据加载器

train_loader = paddle.io.DataLoader(train_dataset, batch_size= 2, shuffle= True)

# 构建测试集数据加载器

valid_loader = paddle.io.DataLoader(train_dataset, batch_size= 2, shuffle= True)

print( '=============train dataset=============')

forLL, LH, HL, HH, label intrain_dataset:

print( 'label: {}'.format(label))

break4、模型训练model2= paddle.Model(model_res)

model2.prepare(optimizer= paddle.optimizer.Adam(parameters=model2.parameters),

loss= nn.CrossEntropyLoss,

metrics= paddle.metric.Accuracy)

model2.fit(train_loader,

valid_loader,

epochs= 5,

verbose= 1,

)总结

本项目主要介绍了如何使用卷积神经网络去检测翻拍图片，主要为摩尔纹图片；其主要创新点在于网络结构上，将图片的高低频信息分开处理。

在本项目中，CNN 仅使用 1 级小波分解进行训练。 可以探索对多级小波分解网络精度的影响。 CNN 模型可以用更多更难的例子和更深的网络进行训练。

ChatGPT数据泄露，技术细节公布 1984 年的电脑也能跑 Chat-GPT ？！ Twitter算法开源

这里有最新开源资讯、软件更新、技术干货等内容

点这里 ↓↓↓ 记得 关注✔ 标星⭐ 哦