3d可视化卷积神经网络

by Chenhua Zhu

朱辰华

引入TensorSpace.js —一种在浏览器中3D可视化神经网络的方法 (Introducing TensorSpace.js — A Way to 3D Visualize Neural Networks in Browsers)

Neural networks were always something high-level, unreachable and mysterious before I took my first deep learning class. To me they were just magic: neural network applications could complete tasks on object detection, image classification and even data prediction in our daily lives.

在我参加第一门深度学习课程之前，神经网络始终是一个高级，不可访问且神秘的事物。 对我来说，它们只是魔术：神经网络应用程序可以完成我们日常生活中的对象检测，图像分类甚至数据预测等任务。

“What does the model compute?” “Why should we use this specific network for this task?” “How could others come up with a structure like this?”

“模型计算什么？” “我们为什么要使用这个特定的网络来完成这项任务？” “其他人怎么能想到这样的结构？”

Maybe you have the same questions as I had. My friends and I have found that sometimes it is really hard to understand and explain neural networks. Then we came up with some ideas:

也许您有与我相同的问题。 我和我的朋友们发现，有时候确实很难理解和解释神经网络。 然后我们提出了一些想法：

Why not visualize a neural network? How about a 3D model? It can be interactive!

为什么不可视化神经网络？ 3D模型怎么样？ 它可以是交互式的！

Since no such thing existed that we could find, we thought, why not create one ourselves? After 6 months, I am proud to introduce our effort: TensorSpace.js.

我们认为，既然不存在我们可以找到的东西，为什么不自己创造一个呢？ 6个月后，我很自豪地介绍了我们的努力： TensorSpace.js 。

什么是TensorSpace.js？ (What is TensorSpace.js?)

TensorSpace.js is a neural network 3D visualization framework built with TensorFlow.js, Three.js and Tween.js.

TensorSpace.js是使用TensorFlow.js，Three.js和Tween.js构建的神经网络3D可视化框架。

Since we wanted to be able to easily present the models in most web browsers, we choose JavaScript to implement the framework.

由于我们希望能够在大多数Web浏览器中轻松呈现模型，因此我们选择JavaScript来实现该框架。

From Fig. 1 above, you can easily check out the model structure: each “cube” represents a “layer” object in the neural network.

从上面的图1，您可以轻松地检查模型结构：每个“立方体”代表神经网络中的一个“层”对象。

Next, you can actually interact with the model by clicking, dragging and scrolling. Different angles may provide different view points of the model. Some objects are expandable, which allows you to observe more details.

接下来，您实际上可以通过单击，拖动和滚动来与模型进行交互。 不同的角度可以提供模型的不同视点。 一些对象是可扩展的，这使您可以观察更多细节。

Furthermore, we designed a hybrid architecture for TensorSpace.js to support different libraries, such as TensorFlow, Keras, and TensorFlow.js (more to come in the future).

此外，我们为TensorSpace.js设计了一种混合体系结构，以支持不同的库，例如TensorFlow，Keras和TensorFlow.js(将来还会有更多)。

TensorSpace.js can not only show the basic model structure, but also present the processes of internal feature abstractions, intermediate data manipulations, and final inference generations.

TensorSpace.js不仅可以显示基本的模型结构，还可以显示内部特征抽象，中间数据操作和最终推断生成的过程。

In summary, TensorSpace.js is:

总而言之，TensorSpace.js是：

• Interactive — Uses Keras-like API to build interactive models in browsers.

  交互式 -使用类似Keras的API在浏览器中构建交互式模型。

• Intuitive — Visualizes the information from intermediate inferences.

  直观 —可视化来自中间推断的信息。

• Integrative — Supports pre-trained models from TensorFlow, Keras, and TensorFlow.js.

  集成 -支持来自TensorFlow，Keras和TensorFlow.js的预训练模型。

TensorSpace.js如何工作？ (How does TensorSpace.js work?)

The following part introduces how to build a TensorSpace model. I’m using an LeNet handwritten recognition model as an example. You can find all the example files from the repo here: TensorSpace — HelloWorld.

以下部分介绍了如何构建TensorSpace模型。 我以LeNet手写识别模型为例。 您可以在以下仓库中找到所有示例文件： TensorSpace-HelloWorld 。

The general workflow is to create or preprocess a pre-trained model with multiple intermediate outputs. Then, based on the neural network structure, we can build a TensorSpace model. Last, we load and initialize the model which can accept input data for inferences.

一般的工作流程是创建或预处理具有多个中间输出的预训练模型。 然后，基于神经网络结构，我们可以构建一个TensorSpace模型。 最后，我们加载并初始化可以接受输入数据进行推理的模型。

After correctly installing TensorSpace.js and properly preprocessing the pre-trained models, we can easily create an LeNet handwritten recognition TensorSpace model. For convenience, we use the preprocessed TensorSpace compatible model and an extracted file which is a handwritten “5” as the model input.

正确安装TensorSpace.js并正确预处理预训练的模型后 ，我们可以轻松创建LeNet手写识别TensorSpace模型。 为方便起见，我们使用预处理过的TensorSpace兼容模型和一个提取的文件 (手写“ 5”)作为模型输入。

let container = document.getElementById( "container" );

// Create sequential modellet model = new TSP.models.Sequential( container );

// Add LeNet Layersmodel.add( new TSP.layers.GreyscaleInput({ shape: [28, 28, 1] }) );model.add( new TSP.layers.Padding2d({ padding: [2, 2] }) );model.add( new TSP.layers.Conv2d({ kernelSize: 5, filters: 6, strides: 1 }) );model.add( new TSP.layers.Pooling2d({ poolSize: [2, 2], strides: [2, 2] }) );model.add( new TSP.layers.Conv2d({ kernelSize: 5, filters: 16, strides: 1 }) );model.add( new TSP.layers.Pooling2d({ poolSize: [2, 2], strides: [2, 2] }) );model.add( new TSP.layers.Dense({ units: 120 }) );model.add( new TSP.layers.Dense({ units: 84 }) );model.add( new TSP.layers.Output1d({    units: 10,    outputs: ["0", "1", "2", "3", "4", "5", "6", "7", "8", "9"]}) );

// Load preprocessed modelmodel.load({    type: "tfjs",    url: './lenetModel/mnist.json'});

// Initialize the model in the browsermodel.init(function() {    // Predict input "5"    model.predict( image_5 ); });

That’s it!

而已！

You can try it out in the CodePen:

您可以在CodePen中试用：

View in CodePen.

在CodePen中查看 。

It is easy to build other preprocessed models in the same way. If you are interested, please check out our playground for more interesting demos, such as Yolov2-tiny, ACGAN, and ResNet-50.

以相同的方式构建其他预处理模型很容易。 如果您有兴趣，请查看我们的游乐场以获得更多有趣的演示，例如Yolov2-tiny，ACGAN和ResNet-50。

什么时候使用？ (When should you use it?)

If you want to present your model to others, explain some detailed features of the model, or build a demo from scratch, I think TensorSpace can be a good tool to describe the model intuitively and clearly. It is fun to interact with and explore the model you just built.

如果您想向其他人展示模型，解释模型的一些详细功能或从头开始构建演示，我认为TensorSpace可以是直观直观地描述模型的好工具。 与您刚建立的模型进行交互和探索很有趣。

结论 (Conclusion)

My team and I hope TensorSpace can, at least, help you move a little step forward on how you visualize neural networks. We believe this will attract more people to this field.

我的团队和我希望TensorSpace至少可以帮助您在可视化神经网络方面迈出一步。 我们相信这将吸引更多的人进入该领域。

For further information about TensorSpace.js, please check out:

有关TensorSpace.js的更多信息，请查看：

• Official Website: TensorSpace.org

  官方网站： TensorSpace.org

• GitHub Repository: TensorSpace-Team/TensorSpace.

  GitHub存储库： TensorSpace-Team / TensorSpace 。

翻译自: https://www.freecodecamp.org/news/tensorspace-js-a-way-to-3d-visualize-neural-networks-in-browsers-2c0afd7648a8/

3d可视化卷积神经网络