文章目录

• 1 前言
• 2 框架数据载体：张量
• 3 Dense数据载体:张量-Tensor
• • 2.1 定义
  • 2.2典型案例
• 4 总结
• 5 感悟

1 前言

大抵来说，从事机器算法框架工作的人员基本有两个比较大的流派。一种是算法派，一直以来都是从事算法方面的工作，算法的理论基础与算法调优经验非常充足，对于算法有较深的理解。另外一种是架构工程派，这类同学以前从事搜、广、推以及大数据等方面的工作，工程架构经验非常丰富，有自己的工程设计理念，对于计算性能有着极致的追求与探索。两种流派并无孰优孰略的问题，最终都会殊路同归，不过由于其研究经历，在学习方式、学习理念上有一定的差异，对于框架的描述也有所不同。

笔者属于先从事工程架构，然后转到算法建模领域，接着又杀回到算法框架领域，所以本章描述会综合算法与工程的思维方式进行阐述。

对于一个大的软件项目来说，有几个比较重要点：

1. 优秀的设计理念如：软件分层、分阶段、良好的灵活性等；
2. 高效的性能：较高的并发支持度QPS、较低的延迟Latency等；
3. 优秀的易用性：可以不通过修改框架代码支持业务目标，较强的包容性；

那么，对于TensorFlow来说，支持声明式的编程，灵活高效，支持预编译等机制，那么他是如何做到的呢。其实大家在上大学的时候，我们学计算机的时候，老师都和我们讲过，程序 = 数据结构 + 算法。那么我们就来看看TensorFlow的数据结构与算法到底有何独到之处。

1. 首先，TensorFlow支持任意的深度模型，要做到这点，需要将原子封装化，包括数据框架化和计算框架化，用白话来说，就是函数的接口要一致，数据的封装接口也要一致，这样才能支持任意的组合。那么对于Tensorflow的计算图来说，Tensor就是这个封装，也可以理解为接口数据的载体。
2. 其次，Tensor作为统一的数据对外封装接口，大多是接口性质的，大多是临时的，所以对于需要常驻存储的数据节点来说我们采用其他的形式，比如Variable，也可以理解为模型数据的载体。
3. 再次，上述已经统一了数据接口，然后对于算子来说，我们也需要统一算子（也就是函数）的接口，并且要保留灵活性，同时要支持用户自己定义的OP算子，也可以理解为模型计算的载体。
4. 再次，通过算法模型同学定义模型结构，组装成计算图，定义出工程架构的计算的DAG。
5. 再次，通过计算图DAG，用会话机制驱动图的运行。

以上完成一个具备“屠龙之技”的深度学习框架。

2 框架数据载体：张量

张量是TensorFlow框架对于整体数据流管理的重要的数据结构，通过提供统一的框架数据载体，从而更加方面的定义数学表达式，更加准确的描述数学模型。在实际的模型运转过程中，模型运转表达式中的数据就有张量来进行承载。TensorFlow提供Tensor和SparseTensor两种张量抽象，分别表示Dense数据与Sparse数据，后者旨在通过设计紧凑的数据节奏解决与优化高维稀疏的内存占比。

3 Dense数据载体:张量-Tensor

从TensorFlow的名字上，其实大家可以看出Tensor是一个非常重要的内容。在TensorFLow中，所有的数据都通过张量的形式对外表达。

2.1 定义

1. 展示形式 ：从数据维度来进行考量，可以理解Tensor为一个N维数组。

• 零阶张量表达标量-Scalar
• 一阶张量表达向量-Vector
• N阶张量表达N维数组

2. 具体实现

从TensorFlow代码的实现来看，Tensor并不直接存储实际的数据，大家可以理解Tensor为一个引用或者是指针，通过对实际数据进行封装，添加一层，对外展示出统一的样式，也就是说Tensor就是一个壳，也可以理解就是一个句柄，他存储的是张量的元信息以及指向实际数据的内存缓冲区的指针。那么为什么采用这样的设计呢。主要是为了实现内存复用，拒绝内存拷贝带来的时空开销。当某个前置操作的输出值被多个后置操作使用的时候，可以通过指针就行多重引用，无需进行重复存储，并且通过引用计数的方式进行数据的生命周期的管控，合理的进行内存调度，类似于Java虚拟机的垃圾回收机制。Tensor的定义代码如下。

• Python实现，代码路径 tensorflow/pyhton/framework/ops.py

@tf_export("Tensor")
class Tensor(_TensorLike):
  """Represents one of the outputs of an `Operation`.

  A `Tensor` is a symbolic handle to one of the outputs of an
  `Operation`. It does not hold the values of that operation's output,
  but instead provides a means of computing those values in a
  TensorFlow `tf.compat.v1.Session`.

  This class has two primary purposes:

  1. A `Tensor` can be passed as an input to another `Operation`.
     This builds a dataflow connection between operations, which
     enables TensorFlow to execute an entire `Graph` that represents a
     large, multi-step computation.

  2. After the graph has been launched in a session, the value of the
     `Tensor` can be computed by passing it to
     `tf.Session.run`.
     `t.eval()` is a shortcut for calling
     `tf.compat.v1.get_default_session().run(t)`.

  In the following example, `c`, `d`, and `e` are symbolic `Tensor`
  objects, whereas `result` is a numpy array that stores a concrete
  value:

  ```python
  # Build a dataflow graph.
  c = tf.constant([[1.0, 2.0], [3.0, 4.0]])
  d = tf.constant([[1.0, 1.0], [0.0, 1.0]])
  e = tf.matmul(c, d)

  # Construct a `Session` to execute the graph.
  sess = tf.compat.v1.Session()

  # Execute the graph and store the value that `e` represents in `result`.
  result = sess.run(e)

  """

  # List of Python operators that we allow to override.
  OVERLOADABLE_OPERATORS = {
      # Binary.
      "__add__",
      "__radd__",
      "__sub__",
      "__rsub__",
      "__mul__",
      "__rmul__",
      "__div__",
      "__rdiv__",
      "__truediv__",
      "__rtruediv__",
      "__floordiv__",
      "__rfloordiv__",
      "__mod__",
      "__rmod__",
      "__lt__",
      "__le__",
      "__gt__",
      "__ge__",
      "__ne__",
      "__eq__",
      "__and__",
      "__rand__",
      "__or__",
      "__ror__",
      "__xor__",
      "__rxor__",
      "__getitem__",
      "__pow__",
      "__rpow__",
      # Unary.
      "__invert__",
      "__neg__",
      "__abs__",
      "__matmul__",
      "__rmatmul__"
  }

  # Whether to allow hashing or numpy-style equality
  _USE_EQUALITY = tf2.enabled()

  def __init__(self, op, value_index, dtype):
    """Creates a new `Tensor`.

    Args:
      op: An `Operation`. `Operation` that computes this tensor.
      value_index: An `int`. Index of the operation's endpoint that produces
        this tensor.
      dtype: A `DType`. Type of elements stored in this tensor.

    Raises:
      TypeError: If the op is not an `Operation`.
    """
    if not isinstance(op, Operation):
      raise TypeError("op needs to be an Operation: %s" % op)
    self._op = op
    self._value_index = value_index
    self._dtype = dtypes.as_dtype(dtype)
    # This will be set by self._as_tf_output().
    self._tf_output = None
    # This will be set by self.shape().
    self._shape_val = None
    # List of operations that use this Tensor as input.  We maintain this list
    # to easily navigate a computation graph.
    self._consumers = []
    self._id = uid()
    self._name = None

  @property
  def op(self):
    """The `Operation` that produces this tensor as an output."""
    return self._op

  @property
  def dtype(self):
    """The `DType` of elements in this tensor."""
    return self._dtype

  @property
  def graph(self):
    """The `Graph` that contains this tensor."""
    return self._op.graph

  @property
  def name(self):
    """The string name of this tensor."""
    if self._name is None:
      if not self._op.name:
        raise ValueError("Operation was not named: %s" % self._op)
      self._name = "%s:%d" % (self._op.name, self._value_index)
    return self._name

  @property
  def device(self):
    """The name of the device on which this tensor will be produced, or None."""
    return self._op.device

  @property
  def shape(self):
    """Returns the `TensorShape` that represents the shape of this tensor.

    The shape is computed using shape inference functions that are
    registered in the Op for each `Operation`.  See
    `tf.TensorShape`
    for more details of what a shape represents.

    The inferred shape of a tensor is used to provide shape
    information without having to launch the graph in a session. This
    can be used for debugging, and providing early error messages. For
    example:

    ```python
    c = tf.constant([[1.0, 2.0, 3.0], [4.0, 5.0, 6.0]])

    print(c.shape)
    ==> TensorShape([Dimension(2), Dimension(3)])

    d = tf.constant([[1.0, 0.0], [0.0, 1.0], [1.0, 0.0], [0.0, 1.0]])

    print(d.shape)
    ==> TensorShape([Dimension(4), Dimension(2)])

    # Raises a ValueError, because `c` and `d` do not have compatible
    # inner dimensions.
    e = tf.matmul(c, d)

    f = tf.matmul(c, d, transpose_a=True, transpose_b=True)

    print(f.shape)
    ==> TensorShape([Dimension(3), Dimension(4)])
    ```

    In some cases, the inferred shape may have unknown dimensions. If
    the caller has additional information about the values of these
    dimensions, `Tensor.set_shape()` can be used to augment the
    inferred shape.

    Returns:
      A `TensorShape` representing the shape of this tensor.

    """
    if self._shape_val is None:
      self._shape_val = self._c_api_shape()
    return self._shape_val
    
    .......

• C++实现，代码路径 tensorflow/core/framework/tensor.h

class Tensor {
 public:
  /// \brief Creates a 1-dimensional, 0-element float tensor.
  ///
  /// The returned Tensor is not a scalar (shape {}), but is instead
  /// an empty one-dimensional Tensor (shape {0}, NumElements() ==
  /// 0). Since it has no elements, it does not need to be assigned a
  /// value and is initialized by default (IsInitialized() is
  /// true). If this is undesirable, consider creating a one-element
  /// scalar which does require initialization:
  ///
  /// ```c++
  ///
  ///     Tensor(DT_FLOAT, TensorShape({}))
  ///
  /// ```
  Tensor();

  /// \brief Creates a Tensor of the given `type` and `shape`.  If
  /// LogMemory::IsEnabled() the allocation is logged as coming from
  /// an unknown kernel and step. Calling the Tensor constructor
  /// directly from within an Op is deprecated: use the
  /// OpKernelConstruction/OpKernelContext allocate_* methods to
  /// allocate a new tensor, which record the kernel and step.
  ///
  /// The underlying buffer is allocated using a `CPUAllocator`.
  Tensor(DataType type, const TensorShape& shape);

  /// \brief Creates a tensor with the input `type` and `shape`, using
  /// the allocator `a` to allocate the underlying buffer. If
  /// LogMemory::IsEnabled() the allocation is logged as coming from
  /// an unknown kernel and step. Calling the Tensor constructor
  /// directly from within an Op is deprecated: use the
  /// OpKernelConstruction/OpKernelContext allocate_* methods to
  /// allocate a new tensor, which record the kernel and step.
  ///
  /// `a` must outlive the lifetime of this Tensor.
  Tensor(Allocator* a, DataType type, const TensorShape& shape);

  /// \brief Creates a tensor with the input `type` and `shape`, using
  /// the allocator `a` and the specified "allocation_attr" to
  /// allocate the underlying buffer. If the kernel and step are known
  /// allocation_attr.allocation_will_be_logged should be set to true
  /// and LogMemory::RecordTensorAllocation should be called after the
  /// tensor is constructed. Calling the Tensor constructor directly
  /// from within an Op is deprecated: use the
  /// OpKernelConstruction/OpKernelContext allocate_* methods to
  /// allocate a new tensor, which record the kernel and step.
  ///
  /// `a` must outlive the lifetime of this Tensor.
  Tensor(Allocator* a, DataType type, const TensorShape& shape,
         const AllocationAttributes& allocation_attr);

  /// \brief Creates a tensor with the input datatype, shape and buf.
  ///
  /// Acquires a ref on buf that belongs to this Tensor.
  Tensor(DataType type, const TensorShape& shape, TensorBuffer* buf);

  /// \brief Creates an empty Tensor of the given data type.
  ///
  /// Like Tensor(), returns a 1-dimensional, 0-element Tensor with
  /// IsInitialized() returning True. See the Tensor() documentation
  /// for details.
  explicit Tensor(DataType type);

 private:
  // A tag type for selecting the `Tensor` constructor overload that creates a
  // scalar tensor in host memory.
  struct host_scalar_tag {};

  class HostScalarTensorBufferBase;
  template <typename T>
  struct ValueAndTensorBuffer;

  // Creates a tensor with the given scalar `value` in CPU memory.
  template <typename T>
  Tensor(T value, host_scalar_tag tag);

 public:
  // A series of specialized constructors for scalar tensors in host memory.
  //
  // NOTE: The `Variant` host-scalar constructor is not defined, because Variant
  // is implicitly constructible from many different types, and this causes
  // ambiguities with some compilers.
  explicit Tensor(float scalar_value)
      : Tensor(scalar_value, host_scalar_tag{}) {}
  explicit Tensor(double scalar_value)
      : Tensor(scalar_value, host_scalar_tag{}) {}
  explicit Tensor(int32 scalar_value)
      : Tensor(scalar_value, host_scalar_tag{}) {}
  explicit Tensor(uint32 scalar_value)
      : Tensor(scalar_value, host_scalar_tag{}) {}
  explicit Tensor(uint16 scalar_value)
      : Tensor(scalar_value, host_scalar_tag{}) {}
  explicit Tensor(uint8 scalar_value)
      : Tensor(scalar_value, host_scalar_tag{}) {}
  explicit Tensor(int16 scalar_value)
      : Tensor(scalar_value, host_scalar_tag{}) {}
  explicit Tensor(int8 scalar_value)
      : Tensor(scalar_value, host_scalar_tag{}) {}
  explicit Tensor(tstring scalar_value)
      : Tensor(std::move(scalar_value), host_scalar_tag{}) {}
  explicit Tensor(complex64 scalar_value)
      : Tensor(scalar_value, host_scalar_tag{}) {}
  explicit Tensor(complex128 scalar_value)
      : Tensor(scalar_value, host_scalar_tag{}) {}
  explicit Tensor(int64 scalar_value)
      : Tensor(scalar_value, host_scalar_tag{}) {}
  explicit Tensor(uint64 scalar_value)
      : Tensor(scalar_value, host_scalar_tag{}) {}
  explicit Tensor(bool scalar_value)
      : Tensor(scalar_value, host_scalar_tag{}) {}
  explicit Tensor(qint8 scalar_value)
      : Tensor(scalar_value, host_scalar_tag{}) {}
  explicit Tensor(quint8 scalar_value)
      : Tensor(scalar_value, host_scalar_tag{}) {}
  explicit Tensor(qint16 scalar_value)
      : Tensor(scalar_value, host_scalar_tag{}) {}
  explicit Tensor(quint16 scalar_value)
      : Tensor(scalar_value, host_scalar_tag{}) {}
  explicit Tensor(qint32 scalar_value)
      : Tensor(scalar_value, host_scalar_tag{}) {}
  explicit Tensor(bfloat16 scalar_value)
      : Tensor(scalar_value, host_scalar_tag{}) {}
  explicit Tensor(Eigen::half scalar_value)
      : Tensor(scalar_value, host_scalar_tag{}) {}
  explicit Tensor(ResourceHandle scalar_value)
      : Tensor(std::move(scalar_value), host_scalar_tag{}) {}

  // NOTE: The `const char*` host-scalar constructor is provided as a
  // convenience because otherwise passing a string literal would surprisingly
  // construct a DT_BOOL tensor.
  explicit Tensor(const char* scalar_value)
      : Tensor(tstring(scalar_value), host_scalar_tag{}) {}

  /// Copy constructor.
  Tensor(const Tensor& other);

  /// \brief Move constructor. After this call, <other> is safely destructible
  /// and can be assigned to, but other calls on it (e.g. shape manipulation)
  /// are not valid.
  Tensor(Tensor&& other);

  ~Tensor();

  /// Returns the data type.
  DataType dtype() const { return shape_.data_type(); }

  /// Returns the shape of the tensor.
  const TensorShape& shape() const { return shape_; }

  /// \brief Convenience accessor for the tensor shape.
  ///
  /// For all shape accessors, see comments for relevant methods of
  /// `TensorShape` in `tensor_shape.h`.
  int dims() const { return shape().dims(); }

  /// Convenience accessor for the tensor shape.
  int64 dim_size(int d) const { return shape().dim_size(d); }

  /// Convenience accessor for the tensor shape.
  int64 NumElements() const { return shape().num_elements(); }

  bool IsSameSize(const Tensor& b) const {
    return shape().IsSameSize(b.shape());
  }

3. 支持的数据类型

2.2典型案例

import tensorflow as tf
t_a = tf.constant([1.0, 2.0], name="t_a") # tf.constant是一个算子，这个计算的结果被张量t_a引用，并且对其生命周期负责
t_b = tf.constant([3.0, 4.0], name="t_b")
t_c = tf.add（t_a, t_b, name="add")
print t_c
输出
Tensor（"add:0", shape(2,), dtype=float32)

从这个示例可以看出，首先TensorFlow的张量与Python中的Numpy的数组是不一样的，他不仅仅有数据，还有一些元信息，其中主要的元信息包括以下三个：

1. 名字 - name：名字是Tensor的唯一标识，同时也表明了这个张量是通过什么方式计算出来的，前面讲解计算图的时候讲过，计算图通过算子的组合与数据的流通进行实现，Tensor就是这个数据的流动，通过Tensor的名字进行标识。Tensor从计算图来说就是算子计算结果，同时算子计算结果可能有多个，所以张量的命名方式可以表示为“node：src_out_index”。其中node为计算这个Tensor节点的名字，src_out_index表示当前的张量是来自计算节点的第几个输出。比如上面的“add:0”就说嘛是计算节点add的第一个输出（与数组下标一致，都是从0开始计数）
2. 维度 - shape：维度是Tensor的形状信息（维度信息），如上述代码所示，比如最后的Tensor t_c的输出是shape=（2，），说明这个张量t_c就是一维数组，长度是2。维度是张量是十分重要的信息，后续涉及到的一些多维数组（矩阵）的操作，都需要维度争取。。
3. 类型 - dtype：类型是Tensor的数据类型信息，每个张量都需要有一个唯一的类型信息，如果不填写，框架会根据数据进行自行推理填充。同时TensorFlow对所有参与运算的张量进行类型检查，如果发现类型不匹配就会进行报错，无法进行。

4 总结

通过上面的描述，可以简单总结下，Tensor基本具备几个作用。

1. 作为整个框架的统一的数据接口
2. 中间结果：实现对中间结果的引用，算子如果看做一个一个节点的话，Tensor及时充当静态数据图在运行是的数据流动管道
3. 最终结果：当计算图构造运行完成后，通过张量可以获取最终的计算结果，也就是最终

下一章节继续分析稀疏张量：SparseTensor，敬请期待。

5 感悟

很多时候，很多情况下，有些事情是非常微妙的，对于一个领域的认知，从不熟悉到得心应手，其实是要花费大量的时间的，但是如果我们很多事情连卖出第一步的勇气都没有，那永远都只能原地踏步。有时候第一步走出去，就成功 一半。

世界上唯一可以不劳而获的就是无知，唯一可以无中生有的就是梦想，虽然世间残酷，但只要你愿意走，总会有路。

想干什么和能干什么是两回事，但是如果连想都不敢想，就不要不说，不要去做了。

标签：__,10,Tensor,17,self,shape,value,scalar,2021
来源： https://blog.csdn.net/qq_22054285/article/details/120813965